Use OpenCV to show camera on android App with correct orientation

Return to OpenCV 4 Android Tutorials List

In this series of tutorials, you are learning OpenCV for Android using Android Studio. So far, you have made a working OpenCV Android Studio Project containing OpenCV module.

In Here is the overview of what we are going to achieve:

  • Create an OpenCV for Android Project (As in previous tutorial in this series) (Here)
  • Show Camera on the Screen
    1. Edit Manifest
      1. Add permission to allow camera.
      2. Support various screen sizes and adaptability of camera.
      3. Use specific features of the camera.
    2. Add a custom Layout show_camera.xml to display camera in it.
    3. Edit MainActivity_show_camera.java 
      1. Import required android classes
      2.  Import OpenCV classes
      3. Connect to OpenCV manager
      4. Initiate LogCat to log events of out app
      5. SHOW CAMERA ON LAYOUT
      6. Corrected Orientation (portrait)

Trust me, it is very simple to do. Lets do it!


 

Manifest.xml

We need to edit the AndroidManifest.xml file like following:

  1. Allow permission to use camera
  2. Allow camera to optimize to the dimensions of the device in use
  3. Use Front camera and autofocus of camera :S

The Manifest.xml file in its entirety is given here

 

This entire process is given here:

Show camera on android App using OpenCV for Android - Edit Manifest

This is all you need to do in AndroidManifest.xml


Create a new layout (show_camera.xml )

Now in your project, do the following:

  1. Create a new layout called show_camera.xml
  2. Add the following code it to:

    Note that the JavaCameraView has the id of show_camera_activity_java_surface_view

This is given below:

 


MainActivity_show_camera.java

Import the following Android Classes in your MainActivity_show_camera:

Import the following OpenCV classes in your MainActivity_show_camera:

Now, lets work with the MainActivity_show_camera Class 🙂 (Code in the end). I will explain every line of OpenCV code in detail.

  • Implement the OpenCV class CvCameraViewListener2 to allow OpenCV to communicate with android camera functionalities.

  • Declare the following items in the class
  • Now, lets call OpenCV manager to help our app communicate with android phone to make OpenCV work

    • CameraBridgeViewBase mOpenCvCameraView
      This variable acts as a bridge between camera and OpenCV library.
    • BaseLoaderCallback mLoaderCallback = …..
      Well, once OpenCV library is loaded, you may want to perform some actions. For example, displaying a success or failure message.
  • Initiate the LogCat
  • Now, when the activity is created, display the OpenCV camera in the layout. show_camera.xml.
  •  The following three functions handle the events when the app is Paused, Resumed and Closed/Destroyed
  • Now, we will do two main things:
    • Receive Image Data when the camera preview starts on your screen 😀
    • Destroy image data when you stop camera preview on your phone screen
    • Now, this one is interesting! OpenCV orients the camera to left by 90 degrees. So if the app is in portrait more, camera will be in -90 or 270 degrees orientation. We fix that in the next and the most important function. There you go!

The entire code is given below:

The entire process of editing MainActivity_show_camera.java is shown here:

Show camera on android App using OpenCV for Android - MainActivity_show_camera


Note: The camera disorientation problem is a bitch an issue not to be takien lightheartedly. Thanks for noting.

This concludes this tutorial, now that you have created a camera app, it is now time to continue this and perform more Computer Vision! In the next tutorial, I will draw a simple object on the screen. Be sure to check out the list of OpenCV for Android tutorials for latest tutorials.

Thanks for reading, and please share and comment if this helped and to share your ideas and opinions.

Return to OpenCV 4 Android Tutorials List

Liked it? Take a second to support codeonion on Patreon!

56 Replies to “Use OpenCV to show camera on android App with correct orientation”

  1. hi thanks for nice tutorial. but i got a problem. code is working fine but the camera view is stretched. i mean it just look like aspect ration issue. what may be the solution for this? thank you.

  2. Hi,

    Thanks for the great blog post. I found it extremely helpful.

    I just wanted to draw to your attention a small issue. I am not sure if I missed something in the instructions, but, here’s the issue: The portrait orientation works fine, however, when rotating the phone to landscape (horizontal) mode, the orientation is now incorrect. The view is also stretched when switching to landscape mode.

    Everything else seems to work okay. Loved the tutorial and the detailed explanations for everything. I would have loved to read about a way to make the portrait mode full-screen.

    Again, thanks for sharing the awesome informational blog post.

  3. Hi,
    Thanks for making this tutorial. 🙂
    I’have a problem, I’m using Android Studio 2.3.3 and I’m using Android emulator (Pixel API 26). When run the application (0 error 0 warning) on emulator appears this message:
    “Package not found
    OpenCV Manager package was not found! Try to install it?”
    If I select “yes”, the message becomes
    “OpenCv Manager
    Package installation failed”

    What could be the problem?
    How can I fix it?

    Thanks 🙂

    1. FATAL EXCEPTION: main
      Process: com.example.junlong.opencv_test, PID: 16704
      java.lang.RuntimeException: Unable to resume activity {com.example.junlong.opencv_test/com.example.junlong.opencv_test.MainActivity_show_camera}: java.lang.IllegalArgumentException: Service Intent must be explicit: Intent { act=org.opencv.engine.BIND }

    2. I changed targetSdkVersion to 19 and it opens. However, it says OpenCV was not initialized correctly. Application will be shut down.

  4. Hello!

    For me by default the show_camera_activity_java_surface_view is weirdly transformed , and has a weird resolution and is not full screen, although the parameters are set to :”fill_parent”.. I tried the full screen orientation solution from one of these comments but that only works if i rotate the device.

    So my question would be that do you know a way to make it full screen by default , so its not transformed weirdly?

    1. Ok well easy fix (sort of — since not exactly what i wanted, but works) i set screen orientation to landscape, and removed the 90 degrees rotation, and now works fine…

      But now i started manipulating the mats, and wanted to save them, but if i use the mainthread, then it does too much work, and receives too many new mats, and if i use a new thread for some reason it never finishes what it should do…

      My guess is that this has something to do with memory management, and the fact that i’m getting waay more mats than required..

      Any idea on how i can reduce the number of mats from the camera?

  5. Hey, great simple example. Helped a lot. Thanks.
    I had one question though, can you toggle flash on/off while you are showing the camera. Or if you could suggest how to set the flash property I can implement the button and toggle myself.
    Thanks again, cheers.

    1. Flash can be controlled once you allow permission

      In manifest,

      Then, you can try something like

      Camera cam;
      void ledon() {
      cam = Camera.open();
      Parameters params = cam.getParameters();
      params.setFlashMode(Parameters.FLASH_MODE_ON);
      cam.setParameters(params);
      cam.startPreview();
      cam.autoFocus(new AutoFocusCallback() {
      public void onAutoFocus(boolean success, Camera camera) {
      }
      });
      }

      void ledoff() {
      cam.stopPreview();
      cam.release();
      }

      I hope it helps

      1. Thanks a lot for your prompt reply.

        I understand the solution you provided. But I’m a little confused on how to implement it. Because the camera is accessed by mOpenCvCameraView which is an object of CameraBridgeViewBase class of OpenCV here and accessing the camera again using a different object (Camera cam in this case) to set flash would return camera already in use.
        Is there a way to set the parameters of flash and then pass the same camera object to the CameraBridgeViewBase class object?

        Sorry for the petty questions, but I am beginner with OpenCV and solving this will really help me a lot. Thanks again.

  6. I am looking forward for taking pictures or saving picture. When you will cover this? Or can you help me a bit with this.

  7. How do I take picture I can only see that my camera is started i cant capture image or snap a photo. Please help me on this.

    1. Thanks for your comment. I did not cover the “Taking pictures” and “Saving in a certain location” in this series yet. Nowadays, I am working on developing the app further. Follow me on twitter and facebook for updates 🙂

  8. for the one who wants provide a full support with all the possible orientations (included reverse-portrait) can use this code below

    @Override
    public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
    mRgba = inputFrame.rgba();
    switch (mOpenCvCameraView.getDisplay().getRotation()) {
    case Surface.ROTATION_0: // Vertical portrait
    Core.transpose(mRgba, mRgbaT);
    Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0);
    Core.flip(mRgbaF, mRgba, 1);
    break;
    case Surface.ROTATION_90: // 90° anti-clockwise
    break;
    case Surface.ROTATION_180: // Vertical anti-portrait
    Core.transpose(mRgba, mRgbaT);
    Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0);
    Core.flip(mRgbaF, mRgba, 0);
    break;
    case Surface.ROTATION_270: // 90° clockwise
    Imgproc.resize(mRgba, mRgbaF, mRgbaF.size(), 0,0, 0);
    Core.flip(mRgbaF, mRgba, -1);
    break;
    default:
    }
    return mRgba;
    }

    don’t forgot to update manifest as well with

    android:screenOrientation=”fullSensor”
    android:configChanges=”keyboardHidden|orientation|screenSize”

  9. Hi there, like some of the other users, I get the following error;

    “It seems that you device does not support camera (or it is locked). Application will be closed.”

    You mentioned in one of your earlier replies that the problem could be in the Manifest file. I’m assuming your referring to AndroidManifest.xml? If so I have the following;

    I have also tried changing the conditions to true but I still the the “It seems that you device does not support camera (or it is locked). Application will be closed.” error message.

    Doesn’t seem to work for either true or false. Can you kindly help point me in the right direction? Thank you.

    1. Try to find your app in the App section of Settings (or something equivalent to your phone) and choose Camera in the Permissions field of your app manually

  10. Hi there,

    I tried importing the following but when I type it out, it immediately disappears;

    import android.view.WindowManager;

    import org.opencv.android.JavaCameraView;
    import org.opencv.android.BaseLoaderCallback;
    import org.opencv.android.CameraBridgeViewBase;
    import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame;
    import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2;
    import org.opencv.android.LoaderCallbackInterface;
    import org.opencv.android.OpenCVLoader;
    import org.opencv.core.Core;
    import org.opencv.core.CvType;
    import org.opencv.core.Mat;
    import org.opencv.imgproc.Imgproc;

    I was able to properly setup the OpenCV project in Android Studio. I’m using Android Studio 2.1.2 on Windows 10.

    1. I was able to rectify this by typing all the code in the final version first. Android Studio 2.1.2 automatically imports the corresponding Android and OpenCV classes.

      In my version I had to modify the onCameraFrame method to the following convention;

      public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {

      // TODO Auto-generated method stub
      mRgba = inputFrame.rgba();
      // Rotate mRgba 90 degrees
      Core.transpose(mRgba, mRgbaT);
      Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0);
      Core.flip(mRgbaF, mRgba, 1 );

      return mRgba; // This function must return
      }

      and

      update -> public MainActivity_show_camera() to public MainActivity() as per my naming convention.

      Hope this helps someone else.

  11. where can i find “MainActivity_show_camera.java” ? or i must add the new mainactivity again ? and how to do that ? please sir help me

    1. Thanks for the comment.

      First of all, it is up to you to rename the main activity. I renamed it to include a customization lesson in the tutorial 🙂

      Basically, if you wish, just rename or create a new activity with the name “MainActivity_show_camera.java”. Also, update Manifest file to reflect the change you made so the activity is loaded (in case if you are willing to make it the first activity that you are loading)

  12. During I run the program, it failed and give some print like this “Service Intent must be explicit: Intent { act=org.opencv.engine.BIND }”, Please help…

  13. hello..
    i used ur code it is good, but i am unable to use camera..it gives me option for dowloading opencv manager.If i download it..code works perfectly otherwise not. what if i want to use org.opencv.android.JavaCameraView without installing opencv manager in my device.

    1. Thanks for the comment :). It is a great idea.

      If that helps, then you can possibly minimize the size of OpenCV manager and provide it for download. I haven’t tried it myself, but you can modify the openCV manager by yourself here https://github.com/Itseez/opencv/blob/master/platforms/android/service/engine/src/org/opencv/engine/manager/ManagerActivity.java

      Or maybe remove the dependancy by adding the required assets in your own project 😀
      Think about creating a lightweight version of OpenCV Manager

      1. Hey Mr codeonion!
        Thanks a lot for sharing this series of tutorials, it is excellent! 🙂

        I also think it is a great idea. It would be great a tutorial about how to include the functionality of the OpenCV Manager directly in apps. Something that explains the following link, but step by step for dummies…
        http://docs.opencv.org/2.4/doc/tutorials/introduction/android_binary_package/dev_with_OCV_on_Android.html#application-development-with-static-initialization

        Why do you think it is said in that link that: “This approach is deprecated for the production code, release package is recommended to communicate with OpenCV Manager via the async initialization described above”?

        btw when are you planning to post the rest of the tutorials (Show a Box on screen, Apply basic color filters using OpenCV, etc) ?

        Cheers!

  14. hii.. thanks for the wonderful tutorial..
    well after the first run am getting a java.lang.nullponiter exception on line
    ” mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);”
    plse help

  15. Hi!
    Nice tutorial and the orientation problem is fixed now. But my preview isn’t full screen, do you know how to handle the problem? Thanks!

  16. Hi! I did everything right but in terms of sending it to a real device( i have samsung galaxy s3 ), i had to decrease the min api level to 16. And when i run the app it doesnt give an error but also nothing happens. canu tell me why? Thanks for the post btw

  17. Hi. Thanks for the post. I did everything correct and I am trying to use it with samsung galaxy s3. When I run the app ,it doesnt fail but also nothing happens too. I decreased my min api to level to 16 to let the app work on samsung s3. Does it effect the working of the app? Thank you

  18. I got this error.

    Camera-JNI: Callback buffer was too small! Expected 1382400 bytes, but got 608256 bytes!

    Please Help.!

  19. I had this problem too. It happens because of Android Marshmallow’s new app permission system.
    You can grant permission manually under Settings > Apps > (your app’s name) > Permissions.

  20. Really good tutorial and easy to follow.
    how can I make it full screen ? and the app give me image with 90 degree orientaion to the left
    do you have any recomendation book should I read or site should I visit to learn opencv4android ?

    1. Hi.

      The orientation issue has been resolved in the code. Now the orientation is working perfectly in portrait mode. You can set it to other orientations too if you wish. Simply go through the tutorial again and you will see that it is now improved, simpler and cleaner.

      Let me know if the problem is solved.

      1. Hi!

        Thanks for the nice tutorials! 🙂
        But i have the full screen issue too, do you already know how to fix the problem?

  21. I get an error after running the app. “It seems that you device does not support camera (or is locked). Application will be closed.” I am running the app on a Nexus 6P with Android 6.0.1. Is this because camera is deprecated? What can I do to fix it?

  22. Thanks for making this tutorial 🙂 My camera is rotated 90 degrees to the left and stays this way even if I rotate my phone. Do you know how to set the camera orientation?

      1. Hi. Thanks for the comment 🙂

        The orientation issue has been resolved in the code. Now the orientation is working perfectly in portrait mode. You can set it to other orientations too if you wish. Simply go through the tutorial again and you will see that it is now improved, simpler and cleaner.

        Let me know if the problem is solved.

    1. Hi. Thanks for the comment 🙂

      The 90 degrees orientation issue has been resolved. Now the orientation is working perfectly in portrait mode. You can set it to other orientations too if you wish. Simply go through the tutorial again and you will see that it is now improved, simpler and cleaner.

      Let me know if the problem is solved.

  23. I use OpenCV 3 and Android Studio 2. Folowing the steps give, I am able to build and run successfully. I had to comment out two lines in MainActivity.java
    //import android.support.design.widget.FloatingActionButton;
    //import android.support.design.widget.Snackbar;
    This temporary fix is applied due to Android Studio not recognising “.design.”.
    Run using USB debugger on Samsung Galaxy S4 Android versioin 5.0.1. First launch this app the user is given the choice to install OpenCV from Apps Store. The correct “ARM” version is automatically located by the device. After installing, re-launch the app to observe that the camera feed is now shown on screen.

    1. Thanks for pointing out 🙂 Those two lines were not necessary in the project. But I refrained myself from editing/removing too much from the default new project code 😀

      I am reflecting the changes in the original post. Thanks 🙂

    1. Thanks for the comment.

      In this tutorial, the class is not used at all. But you can check if it is loaded properly. If you follow this tutorial, you will have the OpenCV libraries and classes loaded properly.

      Make sure that the javaCameraView.java exists in your project browser like this:

      Make sure that javaCemrraView.java exists

      I will make use of this class in the next tutorials. Make sure you read 🙂

      Thanks

Leave a Reply

Your email address will not be published.