Does GrabCut segmentation depend on image size?
I've been thinking about this for quite a while, but haven't really done a detailed analysis of it. Does foreground sharding using the GrabCut [1] algorithm depend on the size of the input image? Intuitively, it seems to me that since the grabcut is based on color models, color distributions should not change as the image is resized, but aliasing artifacts on smaller images can play a role.
Any thoughts or existing experiments on image size versus image segmentation using grabcut would be much appreciated.
thank
[1] C. Rother, V. Kolmogorov, A. Blake, GrabCut: Interactive Foreground Extraction Using Graph Abbreviations, ACM Trans. Chart., Vol. 23, pp. 309-314, 2004.
source to share
Size matters.
The objective function GrabCut balances two members:
- A unary term that measures a pixel according to the foreground / background color model.
- A smoothness term (paired term) that measures the "complexity" of a segmentation boundary.
The first term is (unary) scale with the area in the foreground, while the second (smoothness) is scaled with the perimeter in the foreground.
So, if you scale the image with the x2 factor, you increase the area by x4, and the perimeter is scaled by about the x2 factor.
Therefore, if you have adjusted (or learned) the parameters of the energy function for a specific image size / scale, these parameters may not work for you in different image sizes.
PS
Did you know that Office 2010 "foreground picker" is based on the GrabCut algorithm?
source to share
Here's a PDF of the GrabCut doc , courtesy of Microsoft Research.
The two main effects of image size are runtime and the scale of image detail, which can be considered significant. Of the two, startup time is the one that will bite you with GrabCut - graph processing methods are already quite slow, and GrabCut uses them iteratively.
It is very common to start by downsampling the image to a lower resolution, often in conjunction with a low-pass filter (i.e. you choose the original image with a Gaussian kernel). This greatly reduces the n that the algorithm works while simultaneously reducing the effect of fine detail and noise on the result.
You can also use masking to restrict processing to only certain parts of the image. You already get some of this in GrabCut as an initial "grab" or select step, and then later in the brush processing step. This step also gives you some implicit information about the scale, i.e. The function of interest probably fills most of the selection area.
Recommendation:
Displaying the image at any scale is convenient and reduce the selected area to about n = 100 to 200 k according to their example. If you need to improve the quality of the result, use the result of the initial stage as a starting point for the next iteration at a higher resolution.
source to share