Converting CMSampleBuffer to CVMetalTexture in Swift with CVMetalTextureCacheCreateTextureFromImage
I'm trying to get a simple render-camera-output-to-metal-layer pipeline and it works reasonably well in Objective-C (there's a sample MetalVideoCapture app there), but there seems to be some formatting oddity when I try to translate this quickly. My ultra transparent capture buffer looks like this (ignore lack of sanitation ...)
func captureOutput(captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, fromConnection connection: AVCaptureConnection!) {
var error: CVReturn! = nil
let sourceImageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
let width = CVPixelBufferGetWidth(sourceImageBuffer!)
let height = CVPixelBufferGetHeight(sourceImageBuffer!)
var outTexture: CVMetalTextureRef? = nil
error = CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, videoTextureCache!, sourceImageBuffer!, nil, MTLPixelFormat.BGRA8Unorm, width, height, 0, &outTexture!)
if error != nil {
print("Error! \(error)")
}
let videoTexture = CVMetalTextureGetTexture(outTexture!)
self.imageTexture = videoTexture!
}
Where videoTextureCache var videoTextureCache: CVMetalTextureCache? = nil
But it gives me Cannot invoke 'CVMetalTextureCacheCreateTextureFromImage' with an argument list of type '(CFAllocator!, CVMetalTextureCache, CVImageBuffer, nil, MTLPixelFormat, Int, Int, Int, inout CVMetalTextureRef)'
The thing is, if I replace outTexture with nil, it stops throwing an error, but clearly that won't help me. According to the link for the function, do I need UnsafeMutablePointer? > For this last value. Which I am not sure how to get.
source to share
Try to pre-allocate your textureCache file, this is what I'm using as a member variable:
var _videoTextureCache : Unmanaged<CVMetalTextureCacheRef>?
And then I allocate textureCache in the init method via
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, _context.device, nil, &_videoTextureCache)
where _context.device is MTLDevice. Then, in the captureOutput method, I use the following (be careful, no error checking is included here)
var textureRef : Unmanaged<CVMetalTextureRef>?
CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, _videoTextureCache!.takeUnretainedValue(), imageBuffer, nil, MTLPixelFormat.BGRA8Unorm, width, height, 0, &textureRef)
Hope this helps!
source to share
Update @ peacer212's answer to Swift 3.
You no longer need Unmanaged
and takeUnretainedValue
. So the code should be:
var textureCache: CVMetalTextureCache?
...
CVMetalTextureCacheCreate(kCFAllocatorDefault, nil, _context.device, nil, &_videoTextureCache)
...
var textureRef : CVMetalTexture?
CVMetalTextureCacheCreateTextureFromImage(kCFAllocatorDefault, textureCache!, imageBuffer, nil, .bgra8Unorm, width, height, 0, & textureRef)
source to share