How to use SIFT algorithm with inverted color image
For example, I have two images where the first one is regular and the second one with color inversion (I mean a 255 pixel color value).
I have applied the SIFT algorithm to both of them using OpenCV and Lowe paper , so now I have cue points and descriptors for each image.
The KeyPoints match, but the KeyPoints and Descriptors do not change due to color inversion.
I am wondering if anyone is trying to solve a problem like this?
Also, here's an example of gradients:
I am using OpenCV C ++ implementation using this tutorial and /nonfree/src/sift.cpp modules. Also, I have applied the following method to view gradients:
void MINE::showKeypoints(cv::Mat image, std::vector<cv::KeyPoint> keypoints, string number)
{
cv::Mat img;
image.copyTo(img);
for(int i=0;i<(int)keypoints.size();i++)
{
cv::KeyPoint kp = keypoints[i];
cv::line(img, cv::Point2f(kp.pt.x ,kp.pt.y), cv::Point2f(kp.pt.x ,kp.pt.y), CV_RGB(255,0,0), 4);
cv::line(img, cv::Point2f(kp.pt.x ,kp.pt.y), cv::Point2f(kp.pt.x+kp.size*cos(kp.angle),kp.pt.y+kp.size*sin(kp.angle)), CV_RGB(255,255,0), 1);
}
cv::imshow (str, img);
}
An example of gradients.
As you can see, the gradients of the inverted and original images are not opposite.
source to share
If you negate the input image, then the gradients will have opposite directions ( G <- -G
).
It should be reminded that SIFT descriptors are basically a gradient orientation histogram:
Since the gradient is negative in the inverted image, we get:
-
0th arrow => 4th arrow
-
1st arrow => 5th arrow
-
2nd arrow => 6th arrow
-
3th arrow => 7th arrow
In other words, if you count the first 8-bit histogram (there are 4x4 such histograms in total), and if you designate a
, b
etc. related components of SIFT descriptors, we have:
- original image:
[a, b, c, d, e, f, g, h]
- inverted image:
[e, f, g, h, a, b, c, d]
Thus, you can convert an inverted SIFT descriptor by replacing the components with batches of 4 sizes.
Pseudo-algorithm:
# `sift` is the 128-sized array that represents the descriptor
NCELLS = 16
NORI = 8
0.upto(NCELLS - 1) do |cell|
offset = cell * NORI
offset.upto(offset + NORI/2 - 1) do |i|
sift.swap!(i, i + NORI/2)
end
end
Here's how to check it with vlfeat :
- Cancel default image:
convert -negate default.pgm negate.pgm
- Extract default image keypoints:
./sift --frames default.pgm
- Select the first key point:
tail -n 1 default.frame > kpt.frame
- Describe it with the default image:
./sift --descriptors --read-frames kpt.frame default.pgm
- Describe this with a negative image:
./sift --descriptors --read-frames kpt.frame negate.pgm
- Format both descriptors with 4 components per line (see below)
Then render the output eg. diff -u
or opendiff
: the lines are swapped 2 by 2 as expected.
cat default.descr | ruby -e\
'STDIN.read.split(" ").each_slice(4) {|s| p s}'\
> default.out
cat negate.descr | ruby -e\
'STDIN.read.split(" ").each_slice(4) {|s| p s}'\
> negate.out
source to share
deltheil's answer is correct, but we could easily change the order of 16 to 8 descriptor elements without changing the direction of the gradient (mostly the same thing, but easier to implement)
For example, we have a 2x4 descriptor,
the original was:
[a,b c,d e,f g,h]
the inverted will be:
[g,h e,f c,d a,b]
source to share