Return to OpenCV 4 Android Tutorials List

In this series of tutorials, you are learning OpenCV for Android using Android Studio. So far, you have made a working OpenCV Android Studio Project containing OpenCV module.

In Here is the overview of what we are going to achieve:

• Create an OpenCV for Android Project (As in previous tutorial in this series) (Here)
• Show Camera on the Screen
  1. Edit Manifest
     1. Add permission to allow camera.
     2. Support various screen sizes and adaptability of camera.
     3. Use specific features of the camera.
  2. Add a custom Layout show_camera.xml to display camera in it.
  3. Edit MainActivity_show_camera.java 
     1. Import required android classes
     2.  Import OpenCV classes
     3. Connect to OpenCV manager
     4. Initiate LogCat to log events of out app
     5. SHOW CAMERA ON LAYOUT
     6. Corrected Orientation (portrait)

Trust me, it is very simple to do. Lets do it!

---

 

Manifest.xml

We need to edit the AndroidManifest.xml file like following:

1. Allow permission to use camera
   
   <uses-permission android:name="android.permission.CAMERA"/>

   1	<uses-permission android:name="android.permission.CAMERA"/>

2. Allow camera to optimize to the dimensions of the device in use
   
   <supports-screens android:resizeable="true" android:smallScreens="true" android:normalScreens="true" android:largeScreens="true" android:anyDensity="true" />

   1 2 3 4 5

   <supports-screens android:resizeable="true"         android:smallScreens="true"         android:normalScreens="true"         android:largeScreens="true"         android:anyDensity="true" />

3. Use Front camera and autofocus of camera :S
   
   <uses-feature android:name="android.hardware.camera" android:required="false"/> <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/> <uses-feature android:name="android.hardware.camera.front" android:required="false"/> <uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>

   1 2 3 4

   <uses-feature android:name="android.hardware.camera" android:required="false"/>     <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/>     <uses-feature android:name="android.hardware.camera.front" android:required="false"/>     <uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>

The Manifest.xml file in its entirety is given here

 

This entire process is given here:

This is all you need to do in AndroidManifest.xml

---

Create a new layout (show_camera.xml )

Now in your project, do the following:

1. Create a new layout called show_camera.xml
2. Add the following code it to:
   
   show_camera.xml code

   <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android" android:orientation="vertical" android:layout_width="match_parent" xmlns:opencv="http://schemas.android.com/apk/res-auto" android:layout_height="match_parent"> <org.opencv.android.JavaCameraView android:layout_width="fill_parent" android:layout_height="fill_parent" android:visibility="gone" android:id="@+id/show_camera_activity_java_surface_view" opencv:show_fps="true" opencv:camera_id="any" /> </LinearLayout>

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

   <?xml version="1.0" encoding="utf-8"?> <LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"     android:orientation="vertical" android:layout_width="match_parent"     xmlns:opencv="http://schemas.android.com/apk/res-auto"     android:layout_height="match_parent">       <org.opencv.android.JavaCameraView         android:layout_width="fill_parent"         android:layout_height="fill_parent"         android:visibility="gone"         android:id="@+id/show_camera_activity_java_surface_view"         opencv:show_fps="true"         opencv:camera_id="any" />   </LinearLayout>

   
   Note that the JavaCameraView has the id of show_camera_activity_java_surface_view

This is given below:

 

---

MainActivity_show_camera.java

Import the following Android Classes in your MainActivity_show_camera:

Import the following OpenCV classes in your MainActivity_show_camera:

Now, lets work with the MainActivity_show_camera Class 🙂 (Code in the end). I will explain every line of OpenCV code in detail.

• Implement the OpenCV class CvCameraViewListener2 to allow OpenCV to communicate with android camera functionalities.
  

public class MainActivity extends AppCompatActivity implements CvCameraViewListener2 {

1	public class MainActivity extends AppCompatActivity implements CvCameraViewListener2 {

• Declare the following items in the class
  
  Declaring variables at start of class

  // Used for logging success or failure messages private static final String TAG = "OCVSample::Activity"; // Loads camera view of OpenCV for us to use. This lets us see using OpenCV private CameraBridgeViewBase mOpenCvCameraView; // Used in Camera selection from menu (when implemented) private boolean mIsJavaCamera = true; private MenuItem mItemSwitchCamera = null; // These variables are used (at the moment) to fix camera orientation from 270degree to 0degree Mat mRgba; Mat mRgbaF; Mat mRgbaT;

  1 2 3 4 5 6 7 8 9 10 11 12 13 14

  // Used for logging success or failure messages     private static final String TAG = "OCVSample::Activity";       // Loads camera view of OpenCV for us to use. This lets us see using OpenCV     private CameraBridgeViewBase mOpenCvCameraView;       // Used in Camera selection from menu (when implemented)     private boolean              mIsJavaCamera = true;     private MenuItem             mItemSwitchCamera = null;       // These variables are used (at the moment) to fix camera orientation from 270degree to 0degree     Mat mRgba;     Mat mRgbaF;     Mat mRgbaT;

• Now, lets call OpenCV manager to help our app communicate with android phone to make OpenCV work
  
  Calling OpenCV manager

  private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) { @Override public void onManagerConnected(int status) { switch (status) { case LoaderCallbackInterface.SUCCESS: { Log.i(TAG, "OpenCV loaded successfully"); mOpenCvCameraView.enableView(); } break; default: { super.onManagerConnected(status); } break; } } };

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

  private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {         @Override         public void onManagerConnected(int status) {             switch (status) {                 case LoaderCallbackInterface.SUCCESS:                 {                     Log.i(TAG, "OpenCV loaded successfully");                     mOpenCvCameraView.enableView();                 } break;                 default:                 {                     super.onManagerConnected(status);                 } break;             }         }     };

  • CameraBridgeViewBase mOpenCvCameraView
    This variable acts as a bridge between camera and OpenCV library.
  • BaseLoaderCallback mLoaderCallback = …..
    Well, once OpenCV library is loaded, you may want to perform some actions. For example, displaying a success or failure message.
• Initiate the LogCat
  
  Initiate logcat

  public MainActivity_show_camera() { Log.i(TAG, "Instantiated new " + this.getClass()); }

  1 2 3

  public MainActivity_show_camera() {         Log.i(TAG, "Instantiated new " + this.getClass());     }

• Now, when the activity is created, display the OpenCV camera in the layout. show_camera.xml.
  
  @Override protected void onCreate(Bundle savedInstanceState) { Log.i(TAG, "called onCreate"); super.onCreate(savedInstanceState); getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON); setContentView(R.layout.show_camera); mOpenCvCameraView = (JavaCameraView) findViewById(R.id.show_camera_activity_java_surface_view); mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE); mOpenCvCameraView.setCvCameraViewListener(this); }

  1 2 3 4 5 6 7 8 9 10 11 12 13 14

  @Override     protected void onCreate(Bundle savedInstanceState) {         Log.i(TAG, "called onCreate");         super.onCreate(savedInstanceState);         getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);           setContentView(R.layout.show_camera);           mOpenCvCameraView = (JavaCameraView) findViewById(R.id.show_camera_activity_java_surface_view);           mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);           mOpenCvCameraView.setCvCameraViewListener(this);     }

•  The following three functions handle the events when the app is Paused, Resumed and Closed/Destroyed
  
  @Override public void onPause() { super.onPause(); if (mOpenCvCameraView != null) mOpenCvCameraView.disableView(); } @Override public void onResume() { super.onResume(); if (!OpenCVLoader.initDebug()) { Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization"); OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback); } else { Log.d(TAG, "OpenCV library found inside package. Using it!"); mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS); } } public void onDestroy() { super.onDestroy(); if (mOpenCvCameraView != null) mOpenCvCameraView.disableView(); }

  1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

  @Override     public void onPause()     {         super.onPause();         if (mOpenCvCameraView != null)             mOpenCvCameraView.disableView();     }       @Override     public void onResume()     {         super.onResume();         if (!OpenCVLoader.initDebug()) {             Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");             OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback);         } else {             Log.d(TAG, "OpenCV library found inside package. Using it!");             mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);         }     }       public void onDestroy() {         super.onDestroy();         if (mOpenCvCameraView != null)             mOpenCvCameraView.disableView();     }

• Now, we will do two main things:
  • Receive Image Data when the camera preview starts on your screen 😀
    
    public void onCameraViewStarted(int width, int height) { mRgba = new Mat(height, width, CvType.CV_8UC4); mRgbaF = new Mat(height, width, CvType.CV_8UC4); mRgbaT = new Mat(width, width, CvType.CV_8UC4); }

    1 2 3 4 5 6

    public void onCameraViewStarted(int width, int height) {           mRgba = new Mat(height, width, CvType.CV_8UC4);         mRgbaF = new Mat(height, width, CvType.CV_8UC4);         mRgbaT = new Mat(width, width, CvType.CV_8UC4);     }

  • Destroy image data when you stop camera preview on your phone screen
    
    public void onCameraViewStopped() { mRgba.release(); }

    1 2 3

    public void onCameraViewStopped() {         mRgba.release();     }

  • Now, this one is interesting! OpenCV orients the camera to left by 90 degrees. So if the app is in portrait more, camera will be in -90 or 270 degrees orientation. We fix that in the next and the most important function. There you go!
    
    public Mat onCameraFrame(CvCameraViewFrame inputFrame) { // TODO Auto-generated method stub mRgba = inputFrame.rgba(); // Rotate mRgba 90 degrees Core.transpose(mRgba, mRgbaT); Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0); Core.flip(mRgbaF, mRgba, 1 ); return mRgba; // This function must return }

    1 2 3 4 5 6 7 8 9 10 11

    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {           // TODO Auto-generated method stub         mRgba = inputFrame.rgba();         // Rotate mRgba 90 degrees         Core.transpose(mRgba, mRgbaT);         Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0);         Core.flip(mRgbaF, mRgba, 1 );           return mRgba; // This function must return     }

The entire code is given below:

The entire process of editing MainActivity_show_camera.java is shown here:

---

Note: The camera disorientation problem is a bitch an issue not to be takien lightheartedly. Thanks for noting.

This concludes this tutorial, now that you have created a camera app, it is now time to continue this and perform more Computer Vision! In the next tutorial, I will draw a simple object on the screen. Be sure to check out the list of OpenCV for Android tutorials for latest tutorials.

Thanks for reading, and please share and comment if this helped and to share your ideas and opinions.

Return to OpenCV 4 Android Tutorials List