Why rendering a bitmap with XNA is slow when capturing an image from a webcam, why?

I have been working for several weeks to capture a webcam image and render it in a window shape, but struggled with speed issues all the time. I need at least a 10Hz frame rate to update the background process.

I started using pictureBox, but the solution I ended up with was to create an XNA panel inside my Form and then render the image as a background sprite converting the bitmap to Texture2D using a script I found here.

The problem I am facing now and could not solve it; when i load Bitmap in code by calling bitmap constructor like below everything runs smoothly and i can get high fps. This is what I did during my testing and was very happy with the results.

Bitmap image = new Bitmap(320, 240);

      

But once I submit the bitmap that I take from the webcam, it takes a lot longer to process for some reason and I can't figure it out. As far as I know, bitmaps have the same format, it is only the color of the pixels that are different from each other. What I have checked for format is size (320 * 240), resolution (96) and pixel format (Format32bppArgb). Did I miss something?

This is how I capture the image from the webcam:

VideoCaptureDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);

        FinalVideoSource = new VideoCaptureDevice(VideoCaptureDevices[0].MonikerString);
        FinalVideoSource.DesiredFrameSize = new Size(320, 240);
        FinalVideoSource.DesiredFrameRate = fps;
        FinalVideoSource.NewFrame += new NewFrameEventHandler(FinalVideoSource_NewFrame);

void FinalVideoSource_NewFrame(object sender, NewFrameEventArgs eventArgs)
    {
        // create bitmap from frame
        image = eventArgs.Frame.Clone(new Rectangle(0, 0, 320, 240), PixelFormat.Format32bppArgb);
...

      

This is my paint function in XNA:

protected override void Draw()
    {
        backTexture = GetTexture(GraphicsDevice, image);                

        GraphicsDevice.Clear(Microsoft.Xna.Framework.Color.CornflowerBlue);

        // TODO: Add your drawing code here
        sprites.Begin();
        Vector2 pos = new Vector2(0, 0);
        sprites.Draw(backTexture, pos, Microsoft.Xna.Framework.Color.White);
        sprites.End();
    }


private Texture2D GetTexture(GraphicsDevice dev, System.Drawing.Bitmap bmp)
    {
        int[] imgData = new int[bmp.Width * bmp.Height];
        Texture2D texture = new Texture2D(dev, bmp.Width, bmp.Height);

        unsafe
        {
            // lock bitmap
            System.Drawing.Imaging.BitmapData origdata =
                bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, bmp.PixelFormat);

            uint* byteData = (uint*)origdata.Scan0;

            // Switch bgra -> rgba
            for (int i = 0; i < imgData.Length; i++)
            {
                byteData[i] = (byteData[i] & 0x000000ff) << 16 | (byteData[i] & 0x0000FF00) | (byteData[i] & 0x00FF0000) >> 16 | (byteData[i] & 0xFF000000);
            }

            // copy data
            System.Runtime.InteropServices.Marshal.Copy(origdata.Scan0, imgData, 0, bmp.Width * bmp.Height);

            byteData = null;

            // unlock bitmap
            bmp.UnlockBits(origdata);
        }

        texture.SetData(imgData);

        return texture;
    }

      

I would really appreciate it if someone could help me with this as I am stuck now. The community here has been great and I've managed to get this far without asking before, which is surprising since I have no experience in C # or XNA. With this in mind, I understand that I may be missing something simple or just getting it wrong.

I narrowed it down to loading a bitmap. The only thing I can change when using the newly built bitmap is to simply overwrite it from the webcam before processing it in XNA.

My question now is, am I really missing something, how bitmaps are constructed, which might explain the big difference in rendering speed? Is the conversion to Texture2D the problem here? But I don't see how a different image could affect the speed of this conversion.

+3


source to share


1 answer


Ok. I do not know what's the problem. But I can give you some comments.
First, set your XNA game frame rate equal to or less than your webcam fps. By default, XNA runs at 60fps, so you call GetTexture () twice for each frame on your webcam if you are using 30fps. In the Initialize code:

TargetElapsedTime = TimeSpan.FromSeconds(1f/webcam fps)

      

If that doesn't work ... you can try this code to convert from bitmap to texture.



protected override void Draw()
{
    //Unset the texture from the GraphicsDevice
    for (int i = 0; i < 16; i++)
            {
                if (Game.GraphicsDevice.Textures[i] == backTexture)
                {
                    Game.GraphicsDevice.Textures[i] = null;
                    break;
                }
            }

    backTexture.SetData<byte>(image.GetBytes());                

    GraphicsDevice.Clear(Microsoft.Xna.Framework.Color.CornflowerBlue);

    // TODO: Add your drawing code here
    sprites.Begin();
    Vector2 pos = new Vector2(0, 0);
    sprites.Draw(backTexture, pos, Microsoft.Xna.Framework.Color.White);
    sprites.End();

}


public static byte[] GetBytes(this Bitmap bitmap)
{
    var data = bitmap.LockBits(new System.Drawing.Rectangle(0, 0, bitmap.Width, bitmap.Height), 
        System.Drawing.Imaging.ImageLockMode.ReadOnly, bitmap.PixelFormat);

    // calculate the byte size: for PixelFormat.Format32bppArgb (standard for GDI bitmaps) it the hight * stride
    int bufferSize = data.Height * data.Stride; // stride already incorporates 4 bytes per pixel

    // create buffer
    byte[] bytes = new byte[bufferSize];

    // copy bitmap data into buffer
    Marshal.Copy(data.Scan0, bytes, 0, bytes.Length);

    // unlock the bitmap data
    bitmap.UnlockBits(data);

    return bytes;

}

      

I tested this code and it worked great. Hope for this help.

+1


source







All Articles