Using Camera2 API to get one image and display it using ImageView

I want to use Camera2 API to get one frame from camera and display it using ImageView. I found several related questions like

stackoverflow

I also looked at the Camera2Basic example, but it was too complex and not quite what I needed.

I wrote some code based on some examples I've seen on the internet that should do this, but it doesn't work and I can't figure out why.

The app doesn't crash, but just doesn't display content in the ImageView. I've used log messages in any function call to try and keep the log code transparent.

Also, the logcat app says "the app might be doing too much work in the background .." I can't see how this is possible because I did one captureRequest

instead repeatingCaptureRequest

.

Here's the code and log code: Code:

public class CameraImageReaderActivity extends AppCompatActivity {

private final static String TAG = "CAMERA_IMAGE_READY: ";
private ImageReader imageReader;
private String cameraId;
private CameraDevice camera;
private HandlerThread handlerThread;
private Handler handler;
private Surface imageReaderSurface;
private ImageView imageView;

private CameraDevice.StateCallback cameraStateCallback = new CameraDevice.StateCallback() {
    @Override
    public void onOpened(CameraDevice cameraDevice) {
        Log.d(TAG, "onOpend: CAMERA OPENED");
        camera = cameraDevice;
        getFrames();
    }

    @Override
    public void onDisconnected(CameraDevice cameraDevice) {
        Log.d(TAG, "onDisconnected: CAMERA DISCONNECTED");
        cameraDevice.close();
        camera = null;
    }

    @Override
    public void onError(CameraDevice cameraDevice, int i) {
        Log.d(TAG, "onError: CAMERA ERROR");
        cameraDevice.close();
        camera = null;
    }
};

private CameraCaptureSession.StateCallback captureSessionStateCallback = new CameraCaptureSession.StateCallback() {
    @Override
    public void onConfigured(CameraCaptureSession cameraCaptureSession) {
        Log.d(TAG, "onConfigured: build request and capture");
        try {
            CaptureRequest.Builder requestBuilder = cameraCaptureSession.getDevice().createCaptureRequest(CameraDevice.TEMPLATE_RECORD);
            requestBuilder.addTarget(imageReaderSurface);

            cameraCaptureSession.capture(requestBuilder.build(), null, handler);
        } catch (CameraAccessException e) {
            Log.d(TAG, "onConfigured: CANT CREATE CAPTURE REQUEST");
            e.printStackTrace();
        }
    }

    @Override
    public void onConfigureFailed(CameraCaptureSession cameraCaptureSession) {
        Log.d(TAG, "onConfiguredFailed: CANT CONFIGURE CAMERA");
    }
};

private ImageReader.OnImageAvailableListener imageReaderListener = new ImageReader.OnImageAvailableListener() {
    @Override
    public void onImageAvailable(ImageReader imageReader) {
        Log.d(TAG, "onImageAvailable: IMAGE AVAILABLE");
        Image image = imageReader.acquireLatestImage();
        int imgFormat = image.getFormat();
        ByteBuffer pixelArray1 = image.getPlanes()[0].getBuffer();
        int pixelStride = image.getPlanes()[0].getPixelStride();
        int rowStride = image.getPlanes()[0].getRowStride();
        int rowPadding = rowStride - pixelStride * 640;

        Bitmap bitmap = Bitmap.createBitmap(640 + rowPadding/pixelStride, 480, Bitmap.Config.RGB_565);
        bitmap.copyPixelsFromBuffer(pixelArray1);
        imageView.setImageBitmap(bitmap);

        image.close();
    }
};

/**
 * Sets the cameraId with the front camera id and sets imageReader properties.
 */
public void setupCamera(int width, int height) {
    imageReader = ImageReader.newInstance(width, height, ImageFormat.RGB_565, 30);
    CameraManager cameraManager = (CameraManager) getSystemService(CAMERA_SERVICE);
    try {
        for (String allCamerasId : cameraManager.getCameraIdList()) {
            CameraCharacteristics cameraCharacteristics = cameraManager.getCameraCharacteristics(allCamerasId);
            if (cameraCharacteristics.get(CameraCharacteristics.LENS_FACING) == CameraCharacteristics.LENS_FACING_FRONT) {
                continue;
            }
            cameraId = allCamerasId;
            Log.d(TAG, "setupCamera: CameraId is: " + cameraId);
            return;
        }
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }

}

/**
 * Connects to the front facing camera.
 * After the connection to the camera, the onOpened callback method will be invoked.
 */
public void connectCamera() {
    CameraManager cameraManager = (CameraManager) getSystemService(CAMERA_SERVICE);
    try {
        if (ActivityCompat.checkSelfPermission(this, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
            Log.d(TAG, "CANT OPEN CAMERA");
            // TODO: Consider calling
            //    ActivityCompat#requestPermissions
            // here to request the missing permissions, and then overriding
            //   public void onRequestPermissionsResult(int requestCode, String[] permissions,
            //                                          int[] grantResults)
            // to handle the case where the user grants the permission. See the documentation
            // for ActivityCompat#requestPermissions for more details.
            return;
        }
        cameraManager.openCamera(cameraId, cameraStateCallback, handler);
        Log.d(TAG, "connectCamera: CAMERA OPENED!");
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

/**
 * Build the captureSessionRequest and start in repeat.
 */
public void getFrames() {
    Log.d(TAG, "getFrames: CREATE CAPTURE SESSION");
    imageReaderSurface = imageReader.getSurface();
    List<Surface> surfaceList = new ArrayList<>();
    surfaceList.add(imageReaderSurface);
    try {
        camera.createCaptureSession(surfaceList, captureSessionStateCallback, handler);
    } catch (CameraAccessException e) {
        e.printStackTrace();
    }
}

public void startBackgroundThread() {
    handlerThread = new HandlerThread("CameraImageReaderActivity");
    handlerThread.start();
    handler = new Handler(handlerThread.getLooper());
}

public void stopBackgroundThread() {
    handlerThread.quitSafely();
    try {
        handlerThread.join();
        handlerThread = null;
        handler = null;
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

public void closeCamera() {
    if (camera != null) {
        camera.close();
        camera = null;
    }
}

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_camera_image_reader);
    imageView = (ImageView) findViewById(R.id.imageView);
    setupCamera(640, 480);
    connectCamera();
}

@Override
protected void onPause() {
    closeCamera();
    startBackgroundThread();
    super.onPause();
}

@Override
protected void onResume() {
    super.onResume();
    startBackgroundThread();
    //connectCamera();
}

      

And the (corresponding) logcat:

03-22 14:27:32.900 18806-18806/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: setupCamera: CameraId is: 0
03-22 14:27:32.904 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value mw_continuous-picture
03-22 14:27:32.905 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value emboss
03-22 14:27:32.905 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value sketch
03-22 14:27:32.905 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value neon
03-22 14:27:32.905 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value asd
03-22 14:27:32.905 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value backlight
03-22 14:27:32.905 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value flowers
03-22 14:27:32.905 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value AR
03-22 14:27:32.912 18806-18806/com.example.noamm_000.talkwithcompviawifi I/CameraManager: Using legacy camera HAL.
03-22 14:27:33.685 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value mw_continuous-picture
03-22 14:27:33.685 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value emboss
03-22 14:27:33.685 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value sketch
03-22 14:27:33.685 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value neon
03-22 14:27:33.686 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value asd
03-22 14:27:33.686 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value backlight
03-22 14:27:33.686 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value flowers
03-22 14:27:33.686 18806-18806/com.example.noamm_000.talkwithcompviawifi W/ArrayUtils: Ignoring invalid value AR
03-22 14:27:33.702 18806-18806/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: connectCamera: CAMERA OPENED!
03-22 14:27:33.719 18806-18806/com.example.noamm_000.talkwithcompviawifi I/Choreographer: Skipped 56 frames!  The application may be doing too much work on its main thread.
03-22 14:27:33.787 18806-18806/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onOpend: CAMERA OPENED
03-22 14:27:33.787 18806-18806/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: getFrames: CREATE CAPTURE SESSION
03-22 14:27:33.789 18806-18806/com.example.noamm_000.talkwithcompviawifi I/CameraDeviceState: Legacy camera service transitioning to state CONFIGURING
03-22 14:27:33.789 18806-19149/com.example.noamm_000.talkwithcompviawifi I/RequestThread-0: Configure outputs: 1 surfaces configured.
03-22 14:27:33.790 18806-19149/com.example.noamm_000.talkwithcompviawifi D/Camera: app passed NULL surface
03-22 14:27:33.838 18806-18806/com.example.noamm_000.talkwithcompviawifi I/CameraDeviceState: Legacy camera service transitioning to state IDLE
03-22 14:27:33.843 18806-19150/com.example.noamm_000.talkwithcompviawifi D/CAMERA_IMAGE_READY:: onConfigured: build request and capture
03-22 14:27:33.874 18806-19149/com.example.noamm_000.talkwithcompviawifi W/LegacyRequestMapper: convertRequestMetadata - control.awbRegions setting is not supported, ignoring value
03-22 14:27:33.875 18806-19149/com.example.noamm_000.talkwithcompviawifi W/LegacyRequestMapper: Only received metering rectangles with weight 0.
03-22 14:27:33.875 18806-19149/com.example.noamm_000.talkwithcompviawifi W/LegacyRequestMapper: Only received metering rectangles with weight 0.
03-22 14:27:34.070 18806-18806/com.example.noamm_000.talkwithcompviawifi I/Timeline: Timeline: Activity_idle id: android.os.BinderProxy@13c07070 time:331143683
03-22 14:27:34.317 18806-19155/com.example.noamm_000.talkwithcompviawifi I/CameraDeviceState: Legacy camera service transitioning to state CAPTURING
03-22 14:27:34.353 18806-19149/com.example.noamm_000.talkwithcompviawifi I/CameraDeviceState: Legacy camera service transitioning to state IDLE
03-22 14:27:34.403 18806-18806/com.example.noamm_000.talkwithcompviawifi D/BubblePopupHelper: isShowingBubblePopup : false
03-22 14:27:34.403 18806-18806/com.example.noamm_000.talkwithcompviawifi D/BubblePopupHelper: isShowingBubblePopup : false
03-22 14:27:34.404 18806-18806/com.example.noamm_000.talkwithcompviawifi D/BubblePopupHelper: isShowingBubblePopup : false
03-22 14:27:34.404 18806-18806/com.example.noamm_000.talkwithcompviawifi D/BubblePopupHelper: isShowingBubblePopup : false
03-22 14:28:07.684 18806-18823/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned
03-22 14:28:07.684 18806-18822/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned
03-22 14:28:07.684 18806-19184/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned
03-22 14:28:07.685 18806-18823/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned
03-22 14:28:07.685 18806-18822/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned
03-22 14:28:07.685 18806-19184/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned
03-22 14:28:07.686 18806-18823/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned
03-22 14:28:07.686 18806-18822/com.example.noamm_000.talkwithcompviawifi E/BufferQueueProducer: [unnamed-18806-1] cancelBuffer: BufferQueue has been abandoned

      

Thanks Noam

+3


source to share


3 answers


Try this way to better deal with it on a background thread.



 public void onImageAvailable(ImageReader reader) {
        new ImageSaver(reader.acquireLatestImage());
    } 

   private class ImageSaver implements Runnable {

        private final Image mImage;

        public ImageSaver(Image image) {
            mImage = image;
        }

        @Override
        public void run() {
            File mImageFileName = null;
            if (mImage != null) {

                ByteBuffer byteBuffer = mImage.getPlanes()[0].getBuffer();
                byte[] bytes = new byte[byteBuffer.remaining()];
                byteBuffer.get(bytes);

                FileOutputStream fileOutputStream = null;
                try {
                    mImageFileName = createImageFileName();
                    fileOutputStream = new FileOutputStream(mImageFileName);
                    fileOutputStream.write(bytes);
                } catch (IOException e) {
                    e.printStackTrace();
                } finally {
                    mImage.close();
                    if (mImageFileName != null) {
                        Intent mediaStoreUpdateIntent = new Intent(Intent.ACTION_MEDIA_SCANNER_SCAN_FILE);
                        mediaStoreUpdateIntent.setData(Uri.fromFile(mImageFileName));
                        sendBroadcast(mediaStoreUpdateIntent);
                        loadImageFromStorage(mImageFileName);
                    }
                    if (fileOutputStream != null) {
                        try {
                            fileOutputStream.close();
                        } catch (IOException e) {
                            e.printStackTrace();
                        }
                    }
                }
            }
        }

        private void loadImageFromStorage(File mImageFileName) {
            imageView.setImageBitmap(BitmapFactory.decodeFile(mImageFileName.getAbsolutePath()));
        }
    }

    private File createImageFileName() throws IOException {
        String timestamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
        String prepend = "IMAGE_" + timestamp + "_";
        File imageFile = File.createTempFile(prepend, ".jpg", createImageFolder());
        return imageFile;
    }

    private File createImageFolder() {
        File imageFile = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES);
        File mImageFolder = new File(imageFile, "myFolder");
        if (!mImageFolder.exists()) {
            mImageFolder.mkdirs();
        }
        return mImageFolder;
    }

      

+1


source


Is RGB_565 as the supported format for this camera device, from CameraManager.getCameraCharacteristics (id) .get (CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP) .getOutputFormats ()?

If not, this will generally not work, although it should cause session creation to fail. The only formats guaranteed to be supported are ImageFormat.JPEG and ImageFormat.YUV_420_888.

The former is probably easier to handle (albeit a little less efficient) - copy the JPEG [0] ByteBuffer plane to byte [] and use BitmapFactory.decodeByteArray () to create a bitmap to display.



However, for your particular case, you never seem to call imageReader.setOnImageAvailableListener (imageReaderListener), so you never know about the captured buffer.

Edit: Also, the first few images you get from the camera may be heavily exposed (maybe all black if you're in a poorly lit area) and have poor white balance, focusing, etc. You have to let the camera work for multiple frames so it can adjust the auto exposure / focus / etc to adjust the values ​​before you shoot the final image.

Typically, using a duplicate query with TEMPLATE_PREVIEW and waiting until you see at least CONTROL_AE_STATE_CONVERGED in the capture results would be a good idea; which will still leave focus in a potentially bad state, but whether or not you want to do focusing depends on the use case and output resolution.

+1


source


Since this question is a bit outdated, not sure if it is still relevant, but I'll try to answer.

It creates an instance ImageRader

, ImageReader.OnImageAvailableListener

but is not assigned to your listener ImageReader instance, that you have.

In your method, setupCamera

after instantiating a new ImageReader, set a listener for it:

/**
 * Sets the cameraId with the front camera id and sets imageReader properties.
 */
public void setupCamera(int width, int height) {
    imageReader = ImageReader.newInstance(width, height, ImageFormat.RGB_565, 30);
    // this line is missing
    imageReader.setOnImageAvailableListener(imageReaderListener, handler);
...

      

Hope this helps you or someone else to try your code as a sample.

+1


source







All Articles