Matching grayscale images
I want to check if two images match. Partial matches also interest me. The problem is that images suffer from a lot of noise. Another problem is that images can be rotated at an unknown angle. The objects shown in the images will always have the same scale!
The images show a scan of the area from a top perspective. The "lines" are mainly walls, while other objects are mainly trees and various types of plants.
Another problem was that the left image was very blurry and the correct lines were very thin. To compensate for this difference, I used dilatation. The resulting images are the ones I uploaded.
While it is easy to see that these images are almost perfect, I cannot convince my algorithm of this fact.
My first idea was to match against a function, but the matches are terrible. It only worked for -90 °, 0 ° and 90 ° swing angles. Although most of the descriptors are rotation invariants (they did in past projects), rotation invariance seems to fail for this example.
My second idea was to split the images into several smaller segments and use pattern matching. So I segmented the images, and again, for the human eye, they blend quite easily. The goal of this step was to segment different walls and trees / plants.
The top row is the portions to the left and the bottom row is the portions of the correct image. After segmentation, the segments expanded again.
As mentioned: pattern matching fails, as does path-based pattern matching and path matching.
I think that the expansion of the images was very important because the human eye almost did not match the segments without expansion before segmentation. Another extension after segmentation made it even less difficult.
source to share
Your first job should be to fix the orientation. I'm not sure if this is the best algorithm for this, but here's the approach I would use: fix one of the images and start rotating the other. For each rotation, a histogram is calculated for the intense color on each of the rows / columns. Calculate some distance between the resulting vectors (for example, use a cross product). Choose the turn that results in the smallest cross product. It might be a good idea to combine this approach with hill climbing .
Once you align the images in roughly the same direction, I believe that matching should be easier. Since both images must be at the same scale, calculate something similar to the geometric center for both images: calculate the weighted sum of all pixels - a fully white pixel will have a weight of 1, and a completely black one will have a weight of 0, the sum should be a vector of size 2 (x coordinate and y). Then divide these values by the dimensions of the image and name this "geometric center of the image". Overlay the two images so that the two centers coincide, and then cross-product the difference between the images again. I would say that this must be their difference.
source to share
You can also try the following methods to find rotation and similarity.
-
Use the image moments to get the twist as shown here .
-
After rotating the image, use cross-correlation to evaluate similarity.
EDIT
I tried this with OpenCV and C ++ for two sample images. I am posting the code and results below as it seems to work well, at least for these samples.
Here's a function to calculate the orientation vector using image moments:
Mat orientVec(Mat& im)
{
Moments m = moments(im);
double cov[4] = {m.mu20/m.m00, m.mu11/m.m00, m.mu11/m.m00, m.mu02/m.m00};
Mat covMat(2, 2, CV_64F, cov);
Mat evals, evecs;
eigen(covMat, evals, evecs);
return evecs.row(0);
}
Rotate and Compare Sample Images:
Mat im1 = imread(INPUT_FOLDER_PATH + string("WojUi.png"), 0);
Mat im2 = imread(INPUT_FOLDER_PATH + string("XbrsV.png"), 0);
// get the orientation vector
Mat v1 = orientVec(im1);
Mat v2 = orientVec(im2);
double angle = acos(v1.dot(v2))*180/CV_PI;
// rotate im2. try rotating with -angle and +angle. here using -angle
Mat rot = getRotationMatrix2D(Point(im2.cols/2, im2.rows/2), -angle, 1.0);
Mat im2Rot;
warpAffine(im2, im2Rot, rot, Size(im2.rows, im2.cols));
// add a border to rotated image
int borderSize = im1.rows > im2.cols ? im1.rows/2 + 1 : im1.cols/2 + 1;
Mat im2RotBorder;
copyMakeBorder(im2Rot, im2RotBorder, borderSize, borderSize, borderSize, borderSize,
BORDER_CONSTANT, Scalar(0, 0, 0));
// normalized cross-correlation
Mat& image = im2RotBorder;
Mat& templ = im1;
Mat nxcor;
matchTemplate(image, templ, nxcor, CV_TM_CCOEFF_NORMED);
// take the max
double max;
Point maxPt;
minMaxLoc(nxcor, NULL, &max, NULL, &maxPt);
// draw the match
Mat rgb;
cvtColor(image, rgb, CV_GRAY2BGR);
rectangle(rgb, maxPt, Point(maxPt.x+templ.cols-1, maxPt.y+templ.rows-1), Scalar(0, 255, 255), 2);
cout << "max: " << max << endl;
When pivoting with pivoting in the code, I get max = 0.758. Below is a rotated image in this case with a matching region.
Otherwise max = 0.293
source to share