Introducing absolute differences Stereo-matching algorithm
Good day!
I am trying to find out how to manually implement stereo matching algorithms. I basically start with the most basic ones - Absolute Difference.
I found some slides on the internet that describe how to do this. Basically, from what I understand, I should calculate the difference between the pixels in my left image and the same pixel in the right image, "shifted" by a certain distance / disparity. Then, among these differences, I choose the minimum, which makes sense to me since the pixel with the lowest disparity means that it is most likely the same pixel in the left image.
I prototyped this in MATLAB. Here is the code:
im_left = imread('tsu_left.png');
im_right = imread('tsu_right.png');
height = size(im_left, 1);
width = size(im_left, 2);
disparity_max = 16;
ad_costs = zeros(height, width,disparity_max);
for disparity = 1:disparity_max
for row = 1:height
for col = 1:width
%Left to right matching
col_disp = col - disparity;
if col_disp < 1
ad_costs(row, col, disparity) = 0;
else
%Average RGB
left_pixel = (im_left(row, col, 1) + im_left(row, col, 2) + im_left(row, col, 3))/3;
right_pixel = (im_right(row, col_disp, 1) + im_right(row, col_disp, 2) + im_right(row, col_disp, 3))/3;
%Subtract averages
ad_costs(row, col, disparity) = abs(left_pixel - right_pixel);
end
end
end
end
min_costs = zeros(height, width);
for disparity = 1:disparity_max
for row = 1:height
for col = 1:width
%The minimum disparity is chosen
min_costs(row, col) = min(ad_costs(row, col, :));
end
end
end
Note that I have not implemented an option that sums the differences in a specific window, resulting in a sum of the absolute differences. I only take the difference as a pixel, a mismatch. The lecture slides I found on the internet says it should look like this (rightmost image):
https://dl.dropboxusercontent.com/u/92715312/lec.PNG
However, the result from the code above (using imshow (min_costs)) gives something like this:
https://dl.dropboxusercontent.com/u/92715312/res.PNG
I cannot understand why the outputs are so different. Is there some trivial step I'm missing or my understanding of how the algorithm is not working correctly? I am also using the tsukuba image.
source to share
This is most likely an imshow problem. The imshow function prevents an image from being displayed in the range [0, 255] if it is uint8 or [0.0, 1.0] if it is floating point.
Try:
imshow(min_cost, []);
Note: an empty array for the second argument. This is telling Matlab to figure out the scaling.
Or, use:
imagesc(min_cost); axis image off;
EDIT:
The vanilla rectified stereo with some pixel variety is pretty simple. See the code below:
function [D, C_min, C] = stereo_sad(I1, I2, min_d, max_d, w_radius)
% function [D, C_min, C] = stereo_sad(I1, I2, min_d, max_d, w_radius)
%
% INPUT
% I1 the left stereo image
% I2 the right stereo image
% min_d minimum disparity
% max_d maximum disparity
% w_radius the radius of the window to do the AD aggeration
%
% OUTPUT
% D disparity values
% C_min cost associated with the minimum disparity at pixel (i,j)
% C the cost volume for AD
%
if nargin < 5, w_radius = 4; end % 9x9 window
if nargin < 4, max_d = 64; end
if nargin < 3, min_d = 0; end
% aggregation filter (window size to aggerate the AD cost)
kernel = ones(w_radius*2+1);
kernel = kernel ./ numel(kernel); % normalize it
% grayscale is sufficient for stereo matching
% the green channel is actually a good approximation of the grayscale, we
% could instad do I1 = I1(:,:,2);
if size(I1,3) > 1, I1 = rgb2gray(I1); end
if size(I2,3) > 1, I2 = rgb2gray(I2); end
% conver to double/single
I1 = double(I1);
I2 = double(I2);
% the range of disparity values from min_d to max_d inclusive
d_vals = min_d : max_d;
num_d = length(d_vals);
C = NaN(size(I1,1), size(I1,2), num_d); % the cost volume
% the main loop
for i = 1 : length(d_vals);
d = d_vals(i);
I2_s = imtranslate(I2, [d 0]);
C(:,:,i) = abs(I1 - I2_s); % you could also have SD here (I1-I2_s).^2
C(:,:,i) = imfilter(C(:,:,i), kernel);
end
[C_min, D] = min(C, [], 3);
D = D + min_d;
end
To run the code
I1 = imread (... your left image I2 = imread (... your correct image) D = stereo_sad (I1, I2, 0, 96, 4); imagesc (D); axial image off; Colorbar
You will get a Difference Map as shown below
Steps:
- slide the right image on every mismatch.
- calculate the Absolute Difference between the shifted image and the left image (or some other measure like SSD)
- The average with a rectangular box is the "box" filter.
- Keep averages at volume per pixel
- The mismatch is located at the minimum of the value per pixel. The mismatch will be in the index at the lows
Operations can be performed using Matlab's built-in tools to generate easy-to-read code.
Hope it helps.
source to share
