Use OpenCV to show camera on android App with correct orientation

Return to OpenCV 4 Android Tutorials List

In this series of tutorials, you are learning OpenCV for Android using Android Studio. So far, you have made a working OpenCV Android Studio Project containing OpenCV module.

In Here is the overview of what we are going to achieve:

  • Create an OpenCV for Android Project (As in previous tutorial in this series) (Here)
  • Show Camera on the Screen
    1. Edit Manifest
      1. Add permission to allow camera.
      2. Support various screen sizes and adaptability of camera.
      3. Use specific features of the camera.
    2. Add a custom Layout show_camera.xml to display camera in it.
    3. Edit MainActivity_show_camera.java 
      1. Import required android classes
      2.  Import OpenCV classes
      3. Connect to OpenCV manager
      4. Initiate LogCat to log events of out app
      5. SHOW CAMERA ON LAYOUT
      6. Corrected Orientation (portrait)

Trust me, it is very simple to do. Lets do it!


 

Manifest.xml

We need to edit the AndroidManifest.xml file like following:

  1. Allow permission to use camera
  2. Allow camera to optimize to the dimensions of the device in use
  3. Use Front camera and autofocus of camera :S

The Manifest.xml file in its entirety is given here

 

This entire process is given here:

Show camera on android App using OpenCV for Android - Edit Manifest

This is all you need to do in AndroidManifest.xml


Create a new layout (show_camera.xml )

Now in your project, do the following:

  1. Create a new layout called show_camera.xml
  2. Add the following code it to:

    Note that the JavaCameraView has the id of show_camera_activity_java_surface_view

This is given below:

 


MainActivity_show_camera.java

Import the following Android Classes in your MainActivity_show_camera:

Import the following OpenCV classes in your MainActivity_show_camera:

Now, lets work with the MainActivity_show_camera Class 🙂 (Code in the end). I will explain every line of OpenCV code in detail.

  • Implement the OpenCV class CvCameraViewListener2 to allow OpenCV to communicate with android camera functionalities.

  • Declare the following items in the class
  • Now, lets call OpenCV manager to help our app communicate with android phone to make OpenCV work

    • CameraBridgeViewBase mOpenCvCameraView
      This variable acts as a bridge between camera and OpenCV library.
    • BaseLoaderCallback mLoaderCallback = …..
      Well, once OpenCV library is loaded, you may want to perform some actions. For example, displaying a success or failure message.
  • Initiate the LogCat
  • Now, when the activity is created, display the OpenCV camera in the layout. show_camera.xml.
  •  The following three functions handle the events when the app is Paused, Resumed and Closed/Destroyed
  • Now, we will do two main things:
    • Receive Image Data when the camera preview starts on your screen 😀
    • Destroy image data when you stop camera preview on your phone screen
    • Now, this one is interesting! OpenCV orients the camera to left by 90 degrees. So if the app is in portrait more, camera will be in -90 or 270 degrees orientation. We fix that in the next and the most important function. There you go!

The entire code is given below:

The entire process of editing MainActivity_show_camera.java is shown here:

Show camera on android App using OpenCV for Android - MainActivity_show_camera


Note: The camera disorientation problem is a bitch an issue not to be takien lightheartedly. Thanks for noting.

This concludes this tutorial, now that you have created a camera app, it is now time to continue this and perform more Computer Vision! In the next tutorial, I will draw a simple object on the screen. Be sure to check out the list of OpenCV for Android tutorials for latest tutorials.

Thanks for reading, and please share and comment if this helped and to share your ideas and opinions.

Return to OpenCV 4 Android Tutorials List

Learning the Packages of OpenCV SDK for Android

Return to OpenCV for Android Tutorials List

In the previous tutorial, you learnt about the file structure of OpenCV SDK for Android. Now, we will learn the packages which are available for us to use in OpenCV SDK for Android. At the end of this tutorial, you will definitely know what OpenCV for android has to offer.

Each package is imported like this:

Each packages has java classes which contain functions which you use.

First of all, lets find the packages in the sdk folder manually. On your computer, once you have downloaded the OpenCV SDK, you can browse to the following address in order to find all the OpenCV for Android classes.

OpenCV-android-sdk\sdk\java\src\org\opencv

There, you will find the following packages (OpenCV 3.0):

Also, when you have created an OpenCV project in Android Studio, you will find the packages in the project browser like this:

Now, I will explain to you the purpose of each and every package so that you learn about OpenCV without any effort :D. You won’t regret coming on my blog 🙂

Note that each package can be imported like this as I told before:

Package  Name Purpose Description
android OpenCV interaction with Android platform This package has functions which can allow you to implement an OpenCV project in your Android device. This package handles interactions between OpenCV and Your phone.
calib3d  Project 3D coordinates of objects in a scene. “calib3d” stands for Camera Calibration and 3D Reconstruction. This package has functions which can determine 3D coordinates of objects. So lets say you have a box in a picture. calib3D functions can project 3D coordinates of that box if you provide “intrinsic” and “extrinsic” coordinates.

There are functions to estimate intrinsic and extrinsic parameters 🙂

core Brain of OpenCV. Does all what OpenCV does elsewhere.  “core” is the core package of OpenCV. This package contains functions which provide the base functionality of OpenCV. This include mathematical and matrix operations, algorithmic procedures and much more.

This library is not responsible for image processing. That library is called “imgproc” and it is discussed below.

engine  Provides basic information about current OpenCV SDK installation in your project  Basically, if in your project, you wish to obrain basic information about your OpenCV SDK such as  its packages, version, SDK path, and list of libraries.

Moreover, you can specify a specific version of OpenCV that user’s phone should download from play store. Pretty cool, right?

features2d  Detect and track features in a a 2D test image/frame  You will use this library to detect features, draw them and track them on the screen.
imgcodecs  Read/write images from your disc  Well, imagine you wish to work on an image that is in your disk instead of receiving it from camera. In this case, you will use this library. Its functions allow you to do the following:

  • Read single image
  • Read multiple images
  • Write an image (output)
  • Read specific formats of images in different ways to suit your project and purpose (encode/decode)
imgproc  Image Processing. DUH!  Whether you have loaded an image, or are using camera as input, you will be needing OpenCV to process the image. Well, “imgproc” is the library that has all the functions that you can use, manipulate and exploit to your liking.

This library has functions and classes which allow you to blur image, draw shapes, compare shapes, write text on image, apply enhancement functions and much, much, much more!

This library also has functions which detect points of interest of your liking.

ml  Machine Learning (Artificial Intelligence)  You can train OpenCV to do do Computer Vision according to your requirements. All you need to do is to teach it.

This library has classes and functions that train OpenCV to perform computer vision job for you 🙂

 

objdetect  Detect features or an object  This library has functions that give you the ability to implement detection of a specific custom-defined object in an image or camera feed.

Ever heard of Haar like feature detection? You can do that in this library.

photo  Image Enhancement  This library handles Image enhancement.

Want to set ISO? Exposure setting? contrast? Saturation? hue? Shit? Well, this library has all these functionalities and much more! You can calibrate so much.

One interesting feature is de-noising which will let you play with noise in your image ;D. Useful, right?

utils  Compatibility converters to support different formats of data This library has functions that convert image data from one mathematical format into another. For example, you can convert from one matrix format into another for image processing using a specifically designed code. It is a life saver.
video  Video Processing  This library will help you to perform video processing. You can track objects in a video, subtract background, apply video filters and much more.
videoio  Interact with video files  Using this library, you can open video files, open different format of video files and work with them.

 

This tutorial was brief and was supposed to give you the starting knowledge in order to grow your knowledge about OpenCV by yourself. Let me know in the comments what you think. I will be glad to answer. Now proceed to hte next tutorial 🙂

Return to OpenCV for Android Tutorials List

Android OpenCV Pre-requisites

Return to OpenCV for Android Tutorials List

Welcome to the beginning of the OpenCV for Android (OpenCV4Android) tutorial series. Before we start, you must know that this series expects you to know:

  • Basics of Java
  • XML
  • Computer Vision
  • Object Orient Programming concepts
  • Know how to use Android Studio
  • Know how to make and install an APK file in a real Android device.

Moreover, it is assumed by this series that you have:

  • Downloaded Android Studio.
  • Installed Android Studio.
  • Download Android SDK
  • Installed Android SDK

It may look like too much to ask for, but in reality, a seasoned programmer may already have satisfied the above requirements.

If there is a problem, then let me know, I will sort it out for ya 😉

Return to OpenCV for Android Tutorials List