OpenCV - GrabCut with custom foreground / background models
I want to use the GrabCut algorithm implemented in OpenCV.
As shown in the documentation , this is the function signature:
void grabCut(
InputArray img,
InputOutputArray mask,
Rect rect,
InputOutputArray bgdModel, // *
InputOutputArray fgdModel, // *
int iterCount,
int mode=GC_EVAL)
The mode parameter specifies how to initialize the algorithm, either with rect (rectangular frame) or with a mask (a matrix whose values correspond to custom foreground / background images.
I already have color models for FG and BG, so ideally I don't need to provide a mask or rectangle, but use those models as initializations (I want OpenCV to be able to compute new models and use mine instead). I can see that the bgdModel and fgdModel parameters somehow contain this model information. Unfortunately, the documentation doesn't provide any details on how the model information is stored there.
Is it possible to populate these models with existing data and run the method with mode=GC_EVAL
? if so how do i need to code the models?
source to share
In opencv / sources / modules / imgproc / src / grabcut.cpp you can see how the GMMs are encoded:
GMM::GMM( Mat& _model )
{
const int modelSize = 3/*mean*/ + 9/*covariance*/ + 1/*component weight*/;
if( _model.empty() )
{
_model.create( 1, modelSize*componentsCount, CV_64FC1 );
_model.setTo(Scalar(0));
}
else if( (_model.type() != CV_64FC1) || (_model.rows != 1) || (_model.cols != modelSize*componentsCount) )
CV_Error( CV_StsBadArg, "_model must have CV_64FC1 type, rows == 1 and cols == 13*componentsCount" );
model = _model;
coefs = model.ptr<double>(0);
mean = coefs + componentsCount;
cov = mean + 3*componentsCount;
for( int ci = 0; ci < componentsCount; ci++ )
if( coefs[ci] > 0 )
calcInverseCovAndDeterm( ci );
}
So for each model you need a cv :: Mat of 1 x 65 doubles (componentsCount is 5). There are 3 means per component because it is computed in RGB color space. Using GC_EVAL would really leave the models intact, but I've never tried them with precomputed models.
source to share
Even I had a similar problem. This is how I solved it. I edited the GC_EVAL condition in the grabcut source code for this -
if( mode == GC_EVAL )
{ checkMask( img, mask );
for( int i = 0; i < iterCount; i++ )
{
GCGraph<double> graph;
assignGMMsComponents( img, mask, bgdGMM, fgdGMM, compIdxs );
constructGCGraph(img, mask, bgdGMM, fgdGMM, lambda, leftW, upleftW, upW, uprightW, graph );
estimateSegmentation( graph, mask );
return;
}
}
Note that the learnGMM function is not called here. This is because the foreground and background GMMs are preset.
You can save models in XML file using the following code snippet.
FileStorage fs("mymodels.xml", FileStorage::WRITE);
fs << "BgdModel" << bgdModel;
fs << "FgdModel" << fgdModel;
fs.release();
You can get the models using the following code.
FileStorage fs1("mymodels.xml", FileStorage::READ);
fs1["BgdModel"] >> bgdModel1;
fs1["FgdModel"] >> fgdModel1;
This worked for me.
source to share