Accessing the preview frame from MediaCapture
I would like to capture the preview frames that are displayed inside the CaptureElement
xaml element . source
mine is CaptureElement
set to an object MediaCapture
and I use the method StartPreview()
to start displaying the camera. I would like to access the frames that are displayed without by saving them to an img or video file. The goal is to grab 10 frames per second from the preview and send each frame to a different class that accepts a [] byte .
I tried using the method CapturePhotoToStorageFileAsync
however this is not a workable option as I don't want to take 10 actual images per second. I also don't want to use ScreenCapture
as it stores what is recorded in the video file. Ideally, I don't want to temporarily store media files on my phone. After looking at msdn for MediaCapture
, I noticed there a method called GetPreviewFrameAsync()
, however, this method does not exist inside Windows Phone 8.1.I also came across this example , however I am not quite sure how it works.
Any suggestions on how to approach this would be greatly appreciated.
source to share
There is an example on the Microsoft github page that is relevant, although they target Windows 10. You might be interested in porting your project to get this functionality.
GetPreviewFrame : This sample will capture preview frames rather than full-blown photos. Once he has a preview frame, he can edit the pixels on it.
Here's the relevant part:
private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
// Get information about the preview
var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
// Create the video frame to request a SoftwareBitmap preview frame
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);
// Capture the preview frame
using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame
SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;
// Add a simple green filter effect to the SoftwareBitmap
EditPixels(previewFrame);
}
}
private unsafe void EditPixels(SoftwareBitmap bitmap)
{
// Effect is hard-coded to operate on BGRA8 format only
if (bitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8)
{
// In BGRA8 format, each pixel is defined by 4 bytes
const int BYTES_PER_PIXEL = 4;
using (var buffer = bitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
using (var reference = buffer.CreateReference())
{
// Get a pointer to the pixel buffer
byte* data;
uint capacity;
((IMemoryBufferByteAccess)reference).GetBuffer(out data, out capacity);
// Get information about the BitmapBuffer
var desc = buffer.GetPlaneDescription(0);
// Iterate over all pixels
for (uint row = 0; row < desc.Height; row++)
{
for (uint col = 0; col < desc.Width; col++)
{
// Index of the current pixel in the buffer (defined by the next 4 bytes, BGRA8)
var currPixel = desc.StartIndex + desc.Stride * row + BYTES_PER_PIXEL * col;
// Read the current pixel information into b,g,r channels (leave out alpha channel)
var b = data[currPixel + 0]; // Blue
var g = data[currPixel + 1]; // Green
var r = data[currPixel + 2]; // Red
// Boost the green channel, leave the other two untouched
data[currPixel + 0] = b;
data[currPixel + 1] = (byte)Math.Min(g + 80, 255);
data[currPixel + 2] = r;
}
}
}
}
}
And declare this outside of your class:
[ComImport]
[Guid("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
void GetBuffer(out byte* buffer, out uint capacity);
}
And, of course, your project will have to allow unsafe code for all of this.
Take a closer look at the sample to find out how to get all the details. Or, to walk you through the walkthrough, you can watch a camera session from a recent // build / conference, which includes a small walkthrough of some camera samples.
source to share