Accessing the preview frame from MediaCapture

I would like to capture the preview frames that are displayed inside the CaptureElement

xaml element . source

mine is CaptureElement

set to an object MediaCapture

and I use the method StartPreview()

to start displaying the camera. I would like to access the frames that are displayed without by saving them to an img or video file. The goal is to grab 10 frames per second from the preview and send each frame to a different class that accepts a [] byte .

I tried using the method CapturePhotoToStorageFileAsync

however this is not a workable option as I don't want to take 10 actual images per second. I also don't want to use ScreenCapture

as it stores what is recorded in the video file. Ideally, I don't want to temporarily store media files on my phone. After looking at msdn for MediaCapture

, I noticed there a method called GetPreviewFrameAsync()

, however, this method does not exist inside Windows Phone 8.1.I also came across this example , however I am not quite sure how it works.

Any suggestions on how to approach this would be greatly appreciated.


source to share

1 answer

There is an example on the Microsoft github page that is relevant, although they target Windows 10. You might be interested in porting your project to get this functionality.

GetPreviewFrame : This sample will capture preview frames rather than full-blown photos. Once he has a preview frame, he can edit the pixels on it.

Here's the relevant part:

private async Task GetPreviewFrameAsSoftwareBitmapAsync()
    // Get information about the preview
    var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;

    // Create the video frame to request a SoftwareBitmap preview frame
    var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);

    // Capture the preview frame
    using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
        // Collect the resulting frame
        SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;

        // Add a simple green filter effect to the SoftwareBitmap

private unsafe void EditPixels(SoftwareBitmap bitmap)
    // Effect is hard-coded to operate on BGRA8 format only
    if (bitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8)
        // In BGRA8 format, each pixel is defined by 4 bytes
        const int BYTES_PER_PIXEL = 4;

        using (var buffer = bitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
        using (var reference = buffer.CreateReference())
            // Get a pointer to the pixel buffer
            byte* data;
            uint capacity;
            ((IMemoryBufferByteAccess)reference).GetBuffer(out data, out capacity);

            // Get information about the BitmapBuffer
            var desc = buffer.GetPlaneDescription(0);

            // Iterate over all pixels
            for (uint row = 0; row < desc.Height; row++)
                for (uint col = 0; col < desc.Width; col++)
                    // Index of the current pixel in the buffer (defined by the next 4 bytes, BGRA8)
                    var currPixel = desc.StartIndex + desc.Stride * row + BYTES_PER_PIXEL * col;

                    // Read the current pixel information into b,g,r channels (leave out alpha channel)
                    var b = data[currPixel + 0]; // Blue
                    var g = data[currPixel + 1]; // Green
                    var r = data[currPixel + 2]; // Red

                    // Boost the green channel, leave the other two untouched
                    data[currPixel + 0] = b;
                    data[currPixel + 1] = (byte)Math.Min(g + 80, 255);
                    data[currPixel + 2] = r;


And declare this outside of your class:

unsafe interface IMemoryBufferByteAccess
    void GetBuffer(out byte* buffer, out uint capacity);


And, of course, your project will have to allow unsafe code for all of this.

Take a closer look at the sample to find out how to get all the details. Or, to walk you through the walkthrough, you can watch a camera session from a recent // build / conference, which includes a small walkthrough of some camera samples.



All Articles