Segmentation fault when using MPI and OpenCV at the same time
I am trying to learn MPI in C ++. I have some knowledge of OpenCV, so I tried to write a program using both MPI and OpenCV. It may sound silly, but for educational purposes, I tried to grab an image from a webcam on stream 0 and feed the image to stream 1 to convert to grayscale and display the image in grayscale.
This is how I compile the code: mpic++ opencv.cpp `pkg-config opencv --libs`
The code compiles successfully, but when I run the executable, the image is displayed on the screen for a second and this is what I see in the terminal
~/mpi$ mpirun -np 2 ./a.out
libv4l2: error setting pixformat: Device or resource busy
HIGHGUI ERROR: libv4l unable to ioctl S_FMT
libv4l2: error setting pixformat: Device or resource busy
libv4l1: error setting pixformat: Device or resource busy
HIGHGUI ERROR: libv4l unable to ioctl VIDIOCSPICT
[arch:09670] *** Process received signal ***
[arch:09670] Signal: Segmentation fault (11)
[arch:09670] Signal code: Address not mapped (1)
[arch:09670] Failing at address: 0x218ac50
[arch:09670] [ 0] /usr/lib/libpthread.so.0(+0x10740)[0x7f422fcac740]
[arch:09670] [ 1] /usr/lib/libopencv_core.so.2.4(_ZNK2cv11_InputArray6getMatEi+0x203)[0x7f4233c8c113]
[arch:09670] [ 2] /usr/lib/libopencv_imgproc.so.2.4(_ZN2cv8cvtColorERKNS_11_InputArrayERKNS_12_OutputArrayEii+0x50)[0x7f4232c25de0]
[arch:09670] [ 3] ./a.out[0x408f54]
[arch:09670] [ 4] /usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7f422e9e9800]
[arch:09670] [ 5] ./a.out[0x408c19]
[arch:09670] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 9670 on node arch exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
here is the code
#include <opencv2/opencv.hpp>
#include <mpi.h>
int main(int argc, char **argv) {
cv::Mat_<uint> img(640,480);
cv::Mat_<uint> gray(640,480);
cv::VideoCapture cam(0);
int rank, nproc, j=0;
MPI_Status status;
MPI_Init(&argc, &argv);
// MPI datatype for 8UC3 image
MPI_Datatype mat_8uc3;
MPI_Type_contiguous(sizeof(img), MPI_BYTE, &mat_8uc3);
MPI_Type_commit(&mat_8uc3);
// MPI datatype for 8UC1 image
MPI_Datatype mat_8uc1;
MPI_Type_contiguous(sizeof(gray), MPI_BYTE, &mat_8uc1);
MPI_Type_commit(&mat_8uc1);
MPI_Comm_size(MPI_COMM_WORLD, &nproc); // number of processes
MPI_Comm_rank(MPI_COMM_WORLD, &rank); // rank of the current process
/*
* Thread 0 captures the image from camera
* and sends the image to process 1 for processing
* thread 1 converts the image to grayscale and
* displays the image
*/
if (rank == 0) {
// capture the image and send to thread 1
while (1) {
cam >> img;
cv::imshow("proc 0", img);
MPI_Send(&img, 1, mat_8uc3, 1, j, MPI_COMM_WORLD);
cv::waitKey(40);
j++;
}
}
else if (rank == 1) {
// receive the image, convert to grayscale and display
while (1) {
MPI_Recv(&img, 1, mat_8uc3, 0, j, MPI_COMM_WORLD, &status);
cv::cvtColor(img, gray, cv::COLOR_BGR2GRAY);
cv::imshow("proc 1", gray);
cv::waitKey(20);
j++;
}
}
MPI_Finalize();
return 0;
}
Can someone point out where I am going wrong
thank
EDIT: (after user0815's answer)
Making the suggested changes Device or resource busy
resolved the issue , but the program still segfault.
[arch:01080] *** Process received signal ***
[arch:01080] Signal: Segmentation fault (11)
[arch:01080] Signal code: Address not mapped (1)
[arch:01080] Failing at address: 0x16bbf80
[arch:01080] [ 0] /usr/lib/libpthread.so.0(+0x10740)[0x7fea97322740]
[arch:01080] [ 1] /usr/lib/libopencv_core.so.2.4(_ZNK2cv11_InputArray6getMatEi+0x203)[0x7fea9b302113]
[arch:01080] [ 2] /usr/lib/libopencv_imgproc.so.2.4(_ZN2cv8cvtColorERKNS_11_InputArrayERKNS_12_OutputArrayEii+0x50)[0x7fea9a29bde0]
[arch:01080] [ 3] ./a.out[0x408fc3]
[arch:01080] [ 4] /usr/lib/libc.so.6(__libc_start_main+0xf0)[0x7fea9605f800]
[arch:01080] [ 5] ./a.out[0x408c79]
[arch:01080] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 1080 on node arch exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
source to share
Currently, every process is trying to open the camera. This can cause problems. Try to move the hole to the root section like so:
int main(int argc, char **argv) {
cv::Mat_<uint> img(640,480);
cv::Mat_<uint> gray(640,480);
cv::VideoCapture cam;
/* ... */
if (rank == 0) {
cam.open(0);
/* ... */
}
/* ... */
}
Update:
I think the problem with your code is that you cannot just pass objects with MPI_Send
. Also, the operator sizeof
does not act on objects at all. If you want to transfer an object, you need to transfer the underlying data.
It can be done in your case by sending img.data
with size img.rows * img.cols * sizeof(uint)
. Then you can also use MPI_BYTE as data type and no special types are required.
Some details on the internal structure cv::Mat_
can be found here .
source to share
As @ user0851 pointed out, in your code all processes are trying to open the camera and opening the camera can only be done by the root process.
The openCV object is Mat
quite complex and determining the appropriate one MPI_Datatype
can be tricky as well. Instead, sending an array of pixels is img.data
much easier. Here is a small snippet of code to demonstrate how you can do this. It is compiled mpiCC main.cpp -o main -lopencv_highgui -lopencv_imgproc -lopencv_core
and executedmpirun -np 2 main
#include <opencv2/opencv.hpp>
#include <mpi.h>
using namespace cv;
int main(int argc, char **argv) {
Mat img;
Mat gray;
int rank, nproc, j=0;
size_t total;
size_t elemsize;
int sizes[3];
MPI_Status status;
MPI_Init(&argc, &argv);
MPI_Comm_size(MPI_COMM_WORLD, &nproc); // number of processes
MPI_Comm_rank(MPI_COMM_WORLD, &rank); // rank of the current process
/*
* Thread 0 captures the image from camera
* and sends the image to process 1 for processing
* thread 1 converts the image to grayscale and
* displays the image
*/
if (rank == 0) {
VideoCapture cam(0);
if(!cam.isOpened()){
fprintf(stderr,"unable to open camera.\n");
exit(1);
}
// capture the image and send to thread 1
while (1) {
cam >> img;
cv::imshow("proc 0", img);
if(j==0){
sizes[2]=img.elemSize();
Size s = img.size();
sizes[0] = s.height;
sizes[1] = s.width;
MPI_Send( sizes, 3, MPI_INT, 1,0, MPI_COMM_WORLD);
}
MPI_Send( img.data, sizes[0]*sizes[1]*3, MPI_CHAR,1,1, MPI_COMM_WORLD);
cv::waitKey(40);
j++;
}
}
else if (rank == 1) {
// receive the image, convert to grayscale and display
while (1) {
if(j==0){
MPI_Recv( sizes,3, MPI_INT,0,0, MPI_COMM_WORLD,&status);
img.create(sizes[0],sizes[1],CV_8UC3);
}
MPI_Recv( img.data, sizes[0]*sizes[1]*3, MPI_CHAR,0,1, MPI_COMM_WORLD,&status);
cv::cvtColor(img, gray, cv::COLOR_BGR2GRAY);
cv::imshow("proc 1", gray);
cv::waitKey(20);
j++;
}
}
MPI_Finalize();
return 0;
}
source to share
The Mat object is just a header structure pointing to the memory in which the image is stored. So you have some problems: First, you create Mat
a 640X640 object and than you read from the camera into this object. But Mat
- it's just a header, which is not a pointer to data, the Mat object can now be of any width and height.
Second, it sizeof(Mat)
does not return the amount of memory allocated for the image, but only the amount of memory the Mat object itself. The amount of memory required for an image isMat.total()*Mat.elemSize()
source to share