Emgu CV Surfing detection against a known database?

I am trying to compare an image with a known set of images and find the closest matches using Emgu CV and Surf. I've found a lot of people trying to do the same but not a complete solution that uses the GPU for speed.

The closest thing I got to the tutorial is here:

http://romovs.github.io/blog/2013/07/05/matching-image-to-a-set-of-images-with-emgu-cv/

However, this does not use the GPU and it is very slow for my application. I need something quick like a SurfFeature sample.

So, I tried to refactor this tutorial code according to SurfFeature logic using GPU. Everything went well when GpuMat replaced Matrix here and there. But I ran into a serious problem when I got to the gist of the tutorial above, which is the logic that concatenates all the descriptors into one big matrix. I couldn't find a way to add the GpuMat to each other - even if I could, there is no guarantee that the usual FlannIndex lookup procedure will work even with Gpu based code.

So now I'm stuck on something that I think would be relatively straight forward. For many years people have been trying to do this, so I'm really surprised that there is no published solution.

If you could help me, I would be very grateful. To summarize, I need to do the following:

Create an in-memory (graphical) list of descriptors and cue points for a known set of images using Surf (following the SurfFeature sample). Given an unknown image, search against the material in memory to find the closest match (if any).

Thanks in advance if you can help!