Giter Club home page Giter Club logo

cam2ndk's Introduction

Cam2Ndk

For Machine learning or images processing (OpenCV) applications on Android, it's often divided into two planes: part of it in JAVA and other part in c++(using NDK). There are various Good reasons that such applications should be implemented in native code(c++ planes alone). With NDK Camera 2 APIs released as part of Android 7(API version 24) it has become a blessing for developers and delight for C/C++ lovers.

Example: image processing can be done much efficiently using OpenCV compared to what is available at JAVA plane ( getPixel()/setPixel(), Glide Image Transformation, ColorMatrix, Renderscript )

There are two parts to this repo:

  1. ndk_so: Its a standalone c++ code for camera control and there is no need to use Android Studio (c++ folks would surely love it), just build it from the command line. For this you need to download the version of NDK (cross compilation toolchain for android devices) you need (anything above API version 24) and set the appropriate path in the Makefile. Or if you already have NDK installed using Android Studio? It must be somewhere here -> ~/Library/Android/sdk/ndk/21.2.6472646 (on my macbook).

Once you build the code it will generate lib/libndksamplecam.so. Drop this shared object to jnilibs folder, in ndk-cam project. Currently the Makefile is quite rudimentary and build for ABI type arm64-v8a, similarly more targets can be added.

  1. ndk-cam: This is merely a minimalistic boiler plate code to create textureView and render onto the screen.

Where do we capture Image?

There is a callback function imageCallback inside the object NDKCamera, it gives you captured images for further processing, this image can further be converted to OpenCV Mat object or any other object that your frameworks requires. There is no Java -- C++ boundary to jump to and project delivery/shipping just by delivering shared object .so and JAR is not really required.

NDK Installation:

To know what is NDK and how to install it, please follow the article, for us Step-1 is only required.ย 

OpenCV Integration:

I have integrated OpenCV version 3.4.3, What you need to do is to fetch the OpenCV Android SDK and set the appropriate path for Andoird header files (using -I) and staticlibs (using -L and -l) in the Makefile. In case you need strip down version of OpenCV and not all the modules of OpenCV check out my blog. Mind you, you cannot just pick static .a libray from OpenCV, check this thread

TFLite (.so) for Android:

Check this tensorflow documentation for details as to how to build standalone .so for C++ project. Often TF documentation is broken due to changes, the command to create libtensorflowlite.so is as:

Run ./configure to configure tensorflow WORKSPACE, set it as follows or appropriate:

ANDROID_NDK_HOME=/Users/milinddeore/android-ndk-r14b
ANDROID_NDK_API_LEVEL=24
ANDROID_BUILD_TOOLS_VERSION=29.0.2 
ANDROID_SDK_API_LEVEL=29
ANDROID_SDK_HOME=/Users/milinddeore/library/Android/sdk/

32bit armeabi-v7a:

bazel build -c opt --cxxopt='--std=c++11' --config=monolithic --config=android_arm //tensorflow/lite:libtensorflowlite.so

64bit arm64-v8a:

bazel build -c opt --cxxopt='--std=c++11' --config=monolithic --config=android_arm64 //tensorflow/lite:libtensorflowlite.so

Currently, there is no straightforward way to extract all header files needed, so you must include all header files in tensorflow/lite/ from the TensorFlow repository. Additionally, you will need header files from FlatBuffers and Abseil.

NOTE: only copy the .h files from tensorflow/lite/ folder for compilation and not all the files.

cam2ndk's People

Contributors

milinddeore avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

cam2ndk's Issues

Not so Native ... (NDK)

Hi, I can research and develope some codes in C++, Python, PHP, Java script programing languages and so on easily because I have about 20 years experiment in IT. Recently I decide to research in android codes (Kotlin and Java), But after a week research in I am confused.
The android project which you created here in Native C++ is not so Native, It's confusing ... .
My recent android project in Kotlin is more native!

https://github.com/marzban2030/CameraX-tflite

Device Error Received, code3

I am getting errors on Samsung Galaxy S8

[ImageReader-1280x720f23u3m1-6587-0](id:19bb00000002,api:4,p:1713,c:6587) 
dequeueBuffer: waitForFreeSlotThenRelock: timeout. dequeuedCount: 0, acquiredCount: 0
Device error received, code 3, frame number 10, request ID 0, subseq ID 0

Any ideas?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.