Clever Geek Handbook

Image coordinate with world coordinates

I calibrated my mono camera using opencv. Now I know the internal matrix of the camera and the distortion rates [K1, K2, P1, P2, K3, K4, K5, K6] of my camera. Suppose the camera is at [x, y, z] with rotations [Roll, Pitch, Yaw]. how can i get every pixel in world coordinate when the camera is looking at the floor [z = 0].

enter image description here

+4


source to share


3 answers


You say that you have calibrated your camera, which gives you:

  • Internal parameters
  • External parameters (rotation, translation)
  • Distortion factors

First, to compensate for the distortion, you can use the undistort function and get an undistorted image. Now you have the internal / external parameters and the pinhole camera model. The equation below is taken from the OpenCV documentation explains how to convert 3D world coordinates to 2D image coordinates using these parameters:



enter image description here

Basically, you are multiplying the 3D coordinates by the projection matrix, which in turn is a combination of the intrinsic parameters (the first matrix in the equation) and the extrinsic parameters (the second matrix in the equation). The external parameters matrix contains both rotation and translation components [R|T]

.

+2


source


I suggest you start by examining the pinhole camera model, which simulates the process by which a point in the 3D world is mapped onto an image plane using the camera's internal parameters. As you will see, this process is not individual and therefore usually cannot be inverted (3D image) unless you have depth information (which you do, since you said the points are located at z = 0). This particular case is mentioned on slide 27 of this presentation . The previous lectures describe the imaging process in detail and can be used as a first reference to actually define the transformation of the coordinates of the image to the world. The Szeliski book and this PDF are also great resources.



+2


source


Suppose your camera has a translation T = [xyx] 'according to world reference, and as you said, your camera has R = [roll, pitch yawn] rotation, and your camera parameter is in K. Any pixel ( [px py] on the image plane) has a coordinate W = [X, Y] on the world plane adn W, can be calculated only using the following Matlab code

R = rotationVectorToMatrix(R)'
H=K*[R T];'
Q=inv([H(:,1) H(:,2) -[px;py;1]])*-H(:,4);
W=Q(1:2)

The end of the doc here is a good example of what I mean, https://github.com/muhammetbalcilar/Stereo-Camera-Calibration-Orthogonal-Planes

0


source






More articles:


All Articles