Return to OpenCV 4 Android Tutorials List
In this series of tutorials, you are learning OpenCV for Android using Android Studio. So far, you have made a working OpenCV Android Studio Project containing OpenCV module.
In Here is the overview of what we are going to achieve:
- Create an OpenCV for Android Project (As in previous tutorial in this series) (Here)
- Show Camera on the Screen
- Edit Manifest
- Add permission to allow camera.
- Support various screen sizes and adaptability of camera.
- Use specific features of the camera.
- Add a custom Layout show_camera.xml to display camera in it.
- Edit MainActivity_show_camera.java
- Import required android classes
- Import OpenCV classes
- Connect to OpenCV manager
- Initiate LogCat to log events of out app
- SHOW CAMERA ON LAYOUT
- Corrected Orientation (portrait)
- Edit Manifest
Trust me, it is very simple to do. Lets do it!
Manifest.xml
We need to edit the AndroidManifest.xml file like following:
- Allow permission to use camera
1<uses-permission android:name="android.permission.CAMERA"/> - Allow camera to optimize to the dimensions of the device in use
12345<supports-screens android:resizeable="true"android:smallScreens="true"android:normalScreens="true"android:largeScreens="true"android:anyDensity="true" /> - Use Front camera and autofocus of camera :S
1234<uses-feature android:name="android.hardware.camera" android:required="false"/><uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/><uses-feature android:name="android.hardware.camera.front" android:required="false"/><uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>
The Manifest.xml file in its entirety is given here
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
<?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android="http://schemas.android.com/apk/res/android" package="opencv.codeonion.com.opencv_test"> <application android:allowBackup="true" android:icon="@mipmap/ic_launcher" android:label="@string/app_name" android:supportsRtl="true" android:theme="@style/AppTheme" > <activity android:name=".MainActivity_show_camera" android:label="@string/app_name" android:theme="@style/AppTheme.NoActionBar" android:screenOrientation="portrait"> <intent-filter> <action android:name="android.intent.action.MAIN" /> <category android:name="android.intent.category.LAUNCHER" /> </intent-filter> </activity> </application> <supports-screens android:resizeable="true" android:smallScreens="true" android:normalScreens="true" android:largeScreens="true" android:anyDensity="true" /> <uses-permission android:name="android.permission.CAMERA"/> <uses-feature android:name="android.hardware.camera" android:required="false"/> <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/> <uses-feature android:name="android.hardware.camera.front" android:required="false"/> <uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/> </manifest> |
This entire process is given here:
This is all you need to do in AndroidManifest.xml
Create a new layout (show_camera.xml )
Now in your project, do the following:
- Create a new layout called show_camera.xml
- Add the following code it to:
123456789101112131415<?xml version="1.0" encoding="utf-8"?><LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"android:orientation="vertical" android:layout_width="match_parent"xmlns:opencv="http://schemas.android.com/apk/res-auto"android:layout_height="match_parent"><org.opencv.android.JavaCameraViewandroid:layout_width="fill_parent"android:layout_height="fill_parent"android:visibility="gone"android:id="@+id/show_camera_activity_java_surface_view"opencv:show_fps="true"opencv:camera_id="any" /></LinearLayout>
Note that the JavaCameraView has the id of show_camera_activity_java_surface_view
This is given below:
MainActivity_show_camera.java
Import the following Android Classes in your MainActivity_show_camera:
1 2 3 4 5 6 |
import android.os.Bundle; import android.support.v7.app.AppCompatActivity; import android.util.Log; import android.view.MenuItem; import android.view.SurfaceView; import android.view.WindowManager; |
Import the following OpenCV classes in your MainActivity_show_camera:
1 2 3 4 5 6 7 8 9 10 11 |
import org.opencv.android.JavaCameraView; import org.opencv.android.BaseLoaderCallback; import org.opencv.android.CameraBridgeViewBase; import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame; import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2; import org.opencv.android.LoaderCallbackInterface; import org.opencv.android.OpenCVLoader; import org.opencv.core.Core; import org.opencv.core.CvType; import org.opencv.core.Mat; import org.opencv.imgproc.Imgproc; |
Now, lets work with the MainActivity_show_camera Class 🙂 (Code in the end). I will explain every line of OpenCV code in detail.
- Implement the OpenCV class CvCameraViewListener2 to allow OpenCV to communicate with android camera functionalities.
1 public class MainActivity extends AppCompatActivity implements CvCameraViewListener2 {
- Declare the following items in the class
1234567891011121314// Used for logging success or failure messagesprivate static final String TAG = "OCVSample::Activity";// Loads camera view of OpenCV for us to use. This lets us see using OpenCVprivate CameraBridgeViewBase mOpenCvCameraView;// Used in Camera selection from menu (when implemented)private boolean mIsJavaCamera = true;private MenuItem mItemSwitchCamera = null;// These variables are used (at the moment) to fix camera orientation from 270degree to 0degreeMat mRgba;Mat mRgbaF;Mat mRgbaT; - Now, lets call OpenCV manager to help our app communicate with android phone to make OpenCV work
12345678910111213141516private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {@Overridepublic void onManagerConnected(int status) {switch (status) {case LoaderCallbackInterface.SUCCESS:{Log.i(TAG, "OpenCV loaded successfully");mOpenCvCameraView.enableView();} break;default:{super.onManagerConnected(status);} break;}}};- CameraBridgeViewBase mOpenCvCameraView
This variable acts as a bridge between camera and OpenCV library. - BaseLoaderCallback mLoaderCallback = …..
Well, once OpenCV library is loaded, you may want to perform some actions. For example, displaying a success or failure message.
- CameraBridgeViewBase mOpenCvCameraView
- Initiate the LogCat
123public MainActivity_show_camera() {Log.i(TAG, "Instantiated new " + this.getClass());} - Now, when the activity is created, display the OpenCV camera in the layout. show_camera.xml.
1234567891011121314@Overrideprotected void onCreate(Bundle savedInstanceState) {Log.i(TAG, "called onCreate");super.onCreate(savedInstanceState);getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);setContentView(R.layout.show_camera);mOpenCvCameraView = (JavaCameraView) findViewById(R.id.show_camera_activity_java_surface_view);mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);mOpenCvCameraView.setCvCameraViewListener(this);} - The following three functions handle the events when the app is Paused, Resumed and Closed/Destroyed
1234567891011121314151617181920212223242526@Overridepublic void onPause(){super.onPause();if (mOpenCvCameraView != null)mOpenCvCameraView.disableView();}@Overridepublic void onResume(){super.onResume();if (!OpenCVLoader.initDebug()) {Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback);} else {Log.d(TAG, "OpenCV library found inside package. Using it!");mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);}}public void onDestroy() {super.onDestroy();if (mOpenCvCameraView != null)mOpenCvCameraView.disableView();} - Now, we will do two main things:
- Receive Image Data when the camera preview starts on your screen 😀
123456public void onCameraViewStarted(int width, int height) {mRgba = new Mat(height, width, CvType.CV_8UC4);mRgbaF = new Mat(height, width, CvType.CV_8UC4);mRgbaT = new Mat(width, width, CvType.CV_8UC4);} - Destroy image data when you stop camera preview on your phone screen
123public void onCameraViewStopped() {mRgba.release();} - Now, this one is interesting! OpenCV orients the camera to left by 90 degrees. So if the app is in portrait more, camera will be in -90 or 270 degrees orientation. We fix that in the next and the most important function. There you go!
1234567891011public Mat onCameraFrame(CvCameraViewFrame inputFrame) {// TODO Auto-generated method stubmRgba = inputFrame.rgba();// Rotate mRgba 90 degreesCore.transpose(mRgba, mRgbaT);Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0);Core.flip(mRgbaF, mRgba, 1 );return mRgba; // This function must return}
- Receive Image Data when the camera preview starts on your screen 😀
The entire code is given below:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 |
package opencv.codeonion.com.opencv_test; import android.os.Bundle; import android.support.v7.app.AppCompatActivity; import android.util.Log; import android.view.MenuItem; import android.view.SurfaceView; import android.view.WindowManager; import org.opencv.android.JavaCameraView; import org.opencv.android.BaseLoaderCallback; import org.opencv.android.CameraBridgeViewBase; import org.opencv.android.CameraBridgeViewBase.CvCameraViewFrame; import org.opencv.android.CameraBridgeViewBase.CvCameraViewListener2; import org.opencv.android.LoaderCallbackInterface; import org.opencv.android.OpenCVLoader; import org.opencv.core.Core; import org.opencv.core.CvType; import org.opencv.core.Mat; import org.opencv.imgproc.Imgproc; // OpenCV Classes public class MainActivity_show_camera extends AppCompatActivity implements CvCameraViewListener2 { // Used for logging success or failure messages private static final String TAG = "OCVSample::Activity"; // Loads camera view of OpenCV for us to use. This lets us see using OpenCV private CameraBridgeViewBase mOpenCvCameraView; // Used in Camera selection from menu (when implemented) private boolean mIsJavaCamera = true; private MenuItem mItemSwitchCamera = null; // These variables are used (at the moment) to fix camera orientation from 270degree to 0degree Mat mRgba; Mat mRgbaF; Mat mRgbaT; private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) { @Override public void onManagerConnected(int status) { switch (status) { case LoaderCallbackInterface.SUCCESS: { Log.i(TAG, "OpenCV loaded successfully"); mOpenCvCameraView.enableView(); } break; default: { super.onManagerConnected(status); } break; } } }; public MainActivity_show_camera() { Log.i(TAG, "Instantiated new " + this.getClass()); } @Override protected void onCreate(Bundle savedInstanceState) { Log.i(TAG, "called onCreate"); super.onCreate(savedInstanceState); getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON); setContentView(R.layout.show_camera); mOpenCvCameraView = (JavaCameraView) findViewById(R.id.show_camera_activity_java_surface_view); mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE); mOpenCvCameraView.setCvCameraViewListener(this); } @Override public void onPause() { super.onPause(); if (mOpenCvCameraView != null) mOpenCvCameraView.disableView(); } @Override public void onResume() { super.onResume(); if (!OpenCVLoader.initDebug()) { Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization"); OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, mLoaderCallback); } else { Log.d(TAG, "OpenCV library found inside package. Using it!"); mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS); } } public void onDestroy() { super.onDestroy(); if (mOpenCvCameraView != null) mOpenCvCameraView.disableView(); } public void onCameraViewStarted(int width, int height) { mRgba = new Mat(height, width, CvType.CV_8UC4); mRgbaF = new Mat(height, width, CvType.CV_8UC4); mRgbaT = new Mat(width, width, CvType.CV_8UC4); } public void onCameraViewStopped() { mRgba.release(); } public Mat onCameraFrame(CvCameraViewFrame inputFrame) { // TODO Auto-generated method stub mRgba = inputFrame.rgba(); // Rotate mRgba 90 degrees Core.transpose(mRgba, mRgbaT); Imgproc.resize(mRgbaT, mRgbaF, mRgbaF.size(), 0,0, 0); Core.flip(mRgbaF, mRgba, 1 ); return mRgba; // This function must return } } |
The entire process of editing MainActivity_show_camera.java is shown here:
Note: The camera disorientation problem is a bitch an issue not to be takien lightheartedly. Thanks for noting.
This concludes this tutorial, now that you have created a camera app, it is now time to continue this and perform more Computer Vision! In the next tutorial, I will draw a simple object on the screen. Be sure to check out the list of OpenCV for Android tutorials for latest tutorials.
Thanks for reading, and please share and comment if this helped and to share your ideas and opinions.