Giter Club home page Giter Club logo

oaid / tenginekit Goto Github PK

View Code? Open in Web Editor NEW
2.4K 77.0 315.0 365.15 MB

TengineKit - Free, Fast, Easy, Real-Time Face Detection & Face Landmarks & Face Attributes & Hand Detection & Hand Landmarks & Body Detection & Body Landmarks & Iris Landmarks & Yolov5 SDK On Mobile.

License: Other

C++ 88.56% C 7.43% CMake 0.26% Shell 0.01% Objective-C 0.18% Java 0.83% Makefile 0.69% Kotlin 2.05%
mobile face-landmarks face-detection android face-tracking java face-attributes deep-neural-networks ai artificial-intelligence

tenginekit's Introduction

中文版本

TengineKit

Apache 2.0

TengineKit, developed by OPEN AI LAB.
TengineKit is an easy-to-integrate AI algorithm SDK. At present, it can run on various mobile phones at very low latency.We will continue to update this project for better results and better performance!

Effect

Face Detection &
Face 2dLandmark
Face 3dLandmark &
Iris
Upper Body Detection &
Uppper Body Landmark
Hand Detection &
Hand Landmark

Gif

dance of host

Video( YouTube | BiliBili )

Have a try

  • Apk can be directly downloaded and installed on the phone to see the effect.

or

  • scan code to download apk

Apk

Goals

  • Provide best performance in mobile client
  • Provide the simplest API in mobile client
  • Provide the smallest package in mobile client

Features

  • face detection
  • face landmarks
  • face 3dlandmarks
  • face attributes for example: age, gender, smile, glasses
  • eye iris & landmarks
  • body detect
  • hand detect(Real-time, not yet on Mobile)
  • hand landmarks(Real-time, not yet on Mobile)
  • body detect google(Real-time, not yet on Mobile)
  • body landamrks(Real-time, not yet on Mobile)
  • yolov5

Update (2021/03/25)

  • Fixed Linux sample code errer
  • Update Android sample code, up fps
  • update Linux so file
  • update Linux yolov5s
  • Fixed memory(Core v0.0.6)

Performance(Face Detect & Face Landmark)

CPU Time consuming Frame rate
Kirin 980 4ms 250fps
Qualcomm 855 5ms 200fps
Kirin 970 7ms 142fps
Qualcomm 835 8ms 125fps
Kirin 710F 9ms 111fps
Qualcomm 439 16ms 62fps
MediaTek Helio P60 17ms 59fps
Qualcomm 450B 18ms 56fps

Landmark Points Order

Landmark Points Order

Contact

About the use of TengineKit and face-related technical exchanges, you can join the following QQ groups(Group Answer:TengineKit):

  • TengineKit communication QQ group: 630836519
  • Scan to join group

tenginekit's People

Contributors

bug1989 avatar crusoekid avatar mlbo avatar shitouren1994 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tenginekit's Issues

[Linux] facesdk_readModelFromFile 参数和编译后运行报错问题

如果直接编译 Linux 版本,会出现和 #15 一样的问题。

libTengineKitSDK.so 中,tenginekit_api.h 引用的 facesdk_readModelFromFile 函数的参数和 demo.cpp 中的函数参数不一样,多了一个 bool 参数,我也不知道这个参数的具体意义。

自己在 tenginekit_api.h 加入了这个 bool 参数,比如将

void facesdk_readModelFromFile(ModelType type, const char* model_path, ImageFormat modelInputFormat);

改为

void facesdk_readModelFromFile(ModelType type, const char* model_path, ImageFormat modelInputFormat, bool why);

之后,能够编译成功。但运行识别会报错:

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

colab 复现链接 可以复现这个报错

我看了一下,是 demo.app 的第 45 行会报错:

facesdk_readModelFromFile(ModelType::Detect, "models/detect.bin", ImageFormat::RGB, true);

另外,如果把 45 行的最后那个 bool 参数改为 false 的话:

facesdk_readModelFromFile(ModelType::Detect, "models/detect.bin", ImageFormat::RGB, false);

45 行不会报错,但 46 行会报错:

sdkFaces faces = facesdk_detect(data);

这种情况下,colab 不会显示错误,会直接退出运行,但在我自己机子上会显示这个错误:

Segmentation fault (core dumped)

建议对Cpu使用率或者内存进行优化

目前用的工控板很多都是RK3288或者RK3399,这个sdk在RK3399主板上CPU使用率到达了20%+,然后加上UI绘制,CPU的使用率就比较高了,对比阿里开源的人脸检测SDK,阿里的CPU使用率在RK3399主板上一般是5%左右,希望开源对CPU使用率进行优化

Can I get some info on using the Face 3dLandmark & Iris?

Hello,
Now I want to get the effect like your example in my own image by using TengineKit. The program can generate eye_landmarks and eye_iris, and they all in 3D coordinate, but I have no idea to convert those points to 2D coordinate and draw them on my own image. Should I need to calibrate my camera, so the 3D coordinates points can map to 2D coordinate? Could your please tell me some details about it or give me a demo.
Thanks

未来可能不能用了是么?

"由于jfrog将于2021年5月1日后无法更新,我们无法持续更新SDK库。同时jfrog即将于2022年2月1日停止提供服务,我们的SDK也将在时间点后无法提供正常的服务。"

是否支持对bitmap检测人脸?

代码如下
KitCore.init(context, AndroidConfig
.create()
.setNormalMode()
.openFunc(AndroidConfig.Func.Detect)
.openFunc(AndroidConfig.Func.Attribution)
.setInputImageFormat(AndroidConfig.ImageFormat.RGB));
val detect = Face.detect(bitmap2RGBData(bmp))

或者每次都初始化
KitCore.init(context, AndroidConfig
.create()
.setNormalMode()
.openFunc(AndroidConfig.Func.Detect)
.openFunc(AndroidConfig.Func.Attribution)
.setInputImageFormat(AndroidConfig.ImageFormat.RGB)
.setInputImageSize(bmp.width,bmp.height)
.setOutputImageSize(bmp.width,bmp.height))
val detect = Face.detect(bitmap2RGBData(bmp))

最终返回的faceCount都是0

为啥android上检查图片拿到的人脸都是0?

object THTengineKit: BaseScanCore() {
    init {
        KitCore.init(BaseApplication.application, AndroidConfig
            .create()
            .setDefaultFunc()
            .setNormalMode()
            .setInputImageFormat(AndroidConfig.ImageFormat.RGBA)
        )

        val state = KitCore.getInit()
    }

    override fun doAI(filePath: String): Int {
        val bmp = BitmapFactory.decodeFile(filePath)
        if(bmp != null && !bmp.isRecycled) {
            val faceDetect = Face.detect(getPixelsRGBA(bmp))
//            bmp.recycle()
            val faceDetectInfos = faceDetect.detectInfos
            // 这里一直返回0
            return faceDetectInfos.size
        }
        return 0
    }

    override fun release() {
    }
}

getInit()方法拿到了true,应该是初始化成功的。
getPixelsRGBA方法是closed issue里面的那个方法

fun getPixelsRGBA(image: Bitmap): ByteArray {
        // calculate how many bytes our image consists of
        val bytes = image.byteCount
        val buffer: ByteBuffer = ByteBuffer.allocate(bytes) // Create a new buffer
        image.copyPixelsToBuffer(buffer) // Move the byte data to the buffer
        return buffer.array() // Get the underlying array containing the
}

是哪里出了问题呢???感谢解答

some remarks

Hi! Yout lib is very useful, I'd like to use it in production app, but have some suggestions/remarks:

  1. BlazeFace function should have 6 landmarks besides bounding box, but if I try for single openFunc(AndroidConfig.Func.BlazeFace) make faceDetect.landmark2d(), it will crash.
  2. The lib only have arm .so libs, but doesn't have x86 and x86_64, but google play request all architectures

内存泄漏问题

/zygote: Long monitor contention with owner demo-pool-3 (7295) at com.tenginekit.Face$FaceDetect com.tenginekit.Face.detect(byte[])(Face.java:-1) waiters=2 in com.tenginekit.Face$FaceDetect com.tenginekit.Face.detect(byte[]) for 169ms

linux 编译报错

linux 编译报错

我同样有在编译时错误
执行如下:sudo sh make.sh
[sudo] maphysart 的密码:
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found OpenCV: /usr/local (found version "4.1.2")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/build
Scanning dependencies of target FaceSDKDemo
[ 50%] Building CXX object CMakeFiles/FaceSDKDemo.dir/src/demo.cpp.o
In file included from /home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/src/tenginekit_api.h:7:0,
from /home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/src/demo.cpp:5:
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/src/tenginekit_struct.h:10:1: warning: ‘typedef’ was ignored in this declaration
typedef struct vector2 {
^~~~~~~
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/src/tenginekit_struct.h:52:1: warning: ‘typedef’ was ignored in this declaration
typedef struct BodyInfo
^~~~~~~
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/src/tenginekit_struct.h:58:1: warning: ‘typedef’ was ignored in this declaration
typedef struct HandInfo3d
^~~~~~~
[100%] Linking CXX executable ../bin/FaceSDKDemo
/usr/bin/ld: warning: libcurl.so, needed by /home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so, not found (try using -rpath or -rpath-link)
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_easy_perform’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_easy_getinfo’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_easy_init’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_slist_append’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_version_info’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_easy_cleanup’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_slist_free_all’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_formfree’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_formadd’未定义的引用
/home/maphysart/Desktop/Marks/TengineKit-test/Linux/sample/FaceDemo/libs/libTengineKitSDK.so:对‘curl_easy_setopt’未定义的引用
collect2: error: ld returned 1 exit status
CMakeFiles/FaceSDKDemo.dir/build.make:144: recipe for target '../bin/FaceSDKDemo' failed
make[2]: *** [../bin/FaceSDKDemo] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/FaceSDKDemo.dir/all' failed
make[1]: *** [CMakeFiles/FaceSDKDemo.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

怎样解决?

Is there a python api ??

Glad to see your successful outcomes.

I am doing a project related to face landmark detection, but we use python to realize.
Is there a python api for this project?
thanks

Optimizations, which can be applied for the improvement of the sample code!

First of all, I really like the code and it works pretty fast. I downloaded the sample apk and install to my Android phone. It is super fast. However, when I compile the sample code, it has a little bit delay. Can you explain what kind of optimizations did you use, or if it is possible can you upload the source code of apk?

isProcessingFrame value is on the wrong location!

On the android samples for face, the variable isProcessingFrame on line 115 in the CameraActivity.java, located wrong. It must be set to false after processImage() on line 116. Because when the image processing, the OnPreviewFrame function must be locked. With that correction, fps rate gets a little lower, however, works properly. You can simply change the places of these two lines.

运行Khadas下面的人脸检测和关键点检测出现问题

没有检测结果的输出提示如下:
Tengine plugin allocator TIMVX is registered.
vsiplugin reshape output tensor addr is 0x55a4c31d10
vsiplugin reshape output tensor addr is 0x55a4c31db0
vsiplugin reshape output tensor addr is 0x55a4c31e50
vsiplugin reshape output tensor addr is 0x55a4c31ef0
vsiplugin reshape output tensor addr is 0x55a4c32530
vsiplugin reshape output tensor addr is 0x55a4c320b0
vsiplugin reshape output tensor addr is 0x55a4c32150
vsiplugin reshape output tensor addr is 0x55a4c321f0
TENGINE_LAYOUT_NCHW goto RELEASE
data size is 4472.
data size is 4472.
form : 3 ,to : 0
vsiplugin reshape output tensor addr is 0x55a5af9cc0
TENGINE_LAYOUT_NCHW goto RELEASE
data size is 128.
TENGINE_LAYOUT_NCHW goto RELEASE
data size is 1724.
init Over
Detect Run Graph cost:1.22803
1
Run Over
Tensor size mismatch 256 vs 512.
/home/khadas/projects/khadas/vsiplugin-master/vxnpu/driver/vx_device.c:363 set input data for tensor 0 failed
run subgraph 2 error!
Landmark Run Graph cost:1.41992
Run Over
############################
0,环境Ubuntu20.04,[ 1190.488197] Galcore version 6.4.3.p0.286725
1、采用最新的tengine编译的libtengine-lite.so,采用自带的tm_classification_timvx.c这个函数得到的可执行程序是可以使用npu的。
2、sudo apt install tengine-libs采用这种方式安装,解决了“load plugin failed: libvxplugin.so: cannot open shared object file: No such file or directory”

手部检测数量问题

您好,非常感谢贵团队的工作。

我在使用手部检测 demo 的时候发现,当镜头中同时有两只手 A 和 B 时,只有两只手在镜头中的距离非常非常远的时候,才能在当前帧处理结果中拿到两只手的信息。如若不然,比如相对距离较近(但肯定没有手部遮挡和交叠),就只能拿到一只手的信息,在接下来的帧处理中可能会频繁交替地返回手 A 或者手 B 的信息(但总归只有一只手)。

我认为这一点不是很合理,是做了什么非极大值抑制吗?这一点可以更改优化吗?盼回~

facesdk_handdetect API报错

facesdk_readModelFromFile(ModelType::Detect, "../models/palm_detection_builtin.bin", ImageFormat::RGB); //读取模型 std::cout << "handsdfd:" << std::endl; sdkHand3d hands = facesdk_handdetect(data);//手部检测
使用sdkHand3d hands = facesdk_handdetect(data);//手部检测但是会出现**Segmentation fault (core dumped)**错误?
请问这个如何解决

android deviceid 可以不使用IMEI号吗?

可以使用 MediaDrm 提供的这个id, 除了部分手机恢复出厂设置, 这个id都不变的.
UUID wideVineUuid = new UUID(-0x121074568629b532L, -0x5c37d8232ae2de13L);
wvDrm = new MediaDrm(wideVineUuid);
byte[] wideVineId = wvDrm.getPropertyByteArray(MediaDrm.PROPERTY_DEVICE_UNIQUE_ID);

鸿蒙手机线程中检测人脸,晃动几下很容易出现崩溃ImageResizeHelper::cropYuv

鸿蒙手机,在检测时晃动俩下就会出现如下错误:

2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: Timestamp: 2021-12-02 15:14:38+0800
2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: pid: 32600, tid: 10911, name: Thread-12 >>> com.daofeng.shanghubao <<<
2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: uid: 10808
2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xaff80000
2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: r0 aff80000 r1 b05c2b24 r2 fff7ff7c r3 00000000
2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: r4 b0542ae4 r5 00000008 r6 aff00000 r7 bb6f26a8
2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: r8 b0500000 r9 00000008 r10 000001e0 r11 fffffffc
2021-12-02 15:14:38.595 11099-11099/? A/DEBUG: ip f35344ec sp bb6f2678 lr bcf9d6a7 pc f34d9048
2021-12-02 15:14:38.712 11099-11099/? A/DEBUG: backtrace:
2021-12-02 15:14:38.712 11099-11099/? A/DEBUG: #00 pc 0004f048 /apex/com.android.runtime/lib/bionic/libc.so (__memcpy_a15+112) (BuildId: b3a547af3eb880c5dfd66f61a8c8b4ca)
2021-12-02 15:14:38.712 11099-11099/? A/DEBUG: #1 pc 0003b6a3 /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/lib/arm/libtenginekit_engine.so (ImageResizeHelper::cropYuv(unsigned char const*, unsigned char*, int, int, int, int, int, int)+52) (BuildId: a7eb36802f61e8b75f977c85772960cd9b20aaab)
2021-12-02 15:14:38.712 11099-11099/? A/DEBUG: #2 pc 0001217b /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/lib/arm/libtenginekit_engine.so (faceService::runEyeLandmark(FaceInfo&, unsigned char const*, int, int, bool)+378) (BuildId: a7eb36802f61e8b75f977c85772960cd9b20aaab)
2021-12-02 15:14:38.712 11099-11099/? A/DEBUG: #3 pc 00011d81 /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/lib/arm/libtenginekit_engine.so (faceService::runDetect(unsigned char const*, int, int, std::__ndk1::vector<FaceInfo, std::__ndk1::allocator>&, FaceConfig const&)+332) (BuildId: a7eb36802f61e8b75f977c85772960cd9b20aaab)
2021-12-02 15:14:38.712 11099-11099/? A/DEBUG: #4 pc 0000f077 /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/lib/arm/libtenginekit_engine.so (manager::detectFace(unsigned char const*, std::__ndk1::vector<FaceInfo, std::__ndk1::allocator>&, int, int, FaceConfig const&)+22) (BuildId: a7eb36802f61e8b75f977c85772960cd9b20aaab)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #5 pc 0001b76f /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/lib/arm/libtenginekit_engine.so (Java_com_tenginekit_engine_core_TengineKitEngine_nativeDetectFace+526) (BuildId: a7eb36802f61e8b75f977c85772960cd9b20aaab)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #6 pc 00071853 /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/oat/arm/base.odex (art_jni_trampoline+122)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #7 pc 02009a45 /memfd:/jit-cache (deleted) (com.daofengfun.facevideowh.FFmpegRecordActivity.processImage+1604)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #8 pc 000e3bc5 /apex/com.android.runtime/lib/libart.so (art_quick_invoke_stub_internal+68) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #9 pc 004558bb /apex/com.android.runtime/lib/libart.so (art_quick_invoke_stub+250) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #10 pc 000ebffd /apex/com.android.runtime/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+160) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #11 pc 0022499f /apex/com.android.runtime/lib/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+274) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #12 pc 00220b07 /apex/com.android.runtime/lib/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+802) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #13 pc 0044c96f /apex/com.android.runtime/lib/libart.so (MterpInvokeDirect+358) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #14 pc 000de914 /apex/com.android.runtime/lib/libart.so (mterp_op_invoke_direct+20) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #15 pc 003eaf1c /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/oat/arm/base.vdex (com.daofengfun.facevideowh.FFmpegRecordActivity.access$1400)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #16 pc 0044d2f1 /apex/com.android.runtime/lib/libart.so (MterpInvokeStatic+932) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #17 pc 000de994 /apex/com.android.runtime/lib/libart.so (mterp_op_invoke_static+20) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #18 pc 003ea4ce /data/app/com.daofeng.shanghubao-3GQsNb2-hMt0ZcNIW4Qdcg==/oat/arm/base.vdex (com.daofengfun.facevideowh.FFmpegRecordActivity$ProcessImageThread.run+42)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #19 pc 00200bb3 /apex/com.android.runtime/lib/libart.so (_ZN3art11interpreterL7ExecuteEPNS_6ThreadERKNS_20CodeItemDataAccessorERNS_11ShadowFrameENS_6JValueEbb.llvm.14250088896784669869+266) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #20 pc 00205465 /apex/com.android.runtime/lib/libart.so (art::interpreter::EnterInterpreterFromEntryPoint(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*)+120) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #21 pc 0043ed29 /apex/com.android.runtime/lib/libart.so (artQuickToInterpreterBridge+832) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #22 pc 000e85a1 /apex/com.android.runtime/lib/libart.so (art_quick_to_interpreter_bridge+32) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #23 pc 000e3bc5 /apex/com.android.runtime/lib/libart.so (art_quick_invoke_stub_internal+68) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #24 pc 004558bb /apex/com.android.runtime/lib/libart.so (art_quick_invoke_stub+250) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #25 pc 000ebffd /apex/com.android.runtime/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+160) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #26 pc 003932bb /apex/com.android.runtime/lib/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+54) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #27 pc 00393fc5 /apex/com.android.runtime/lib/libart.so (art::InvokeVirtualOrInterfaceWithJValues(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, jvalue const*)+300) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #28 pc 003c7f0b /apex/com.android.runtime/lib/libart.so (art::Thread::CreateCallback(void*)+982) (BuildId: f19139b4309180c887ec6b2ca9af235c)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #29 pc 0009c5e7 /apex/com.android.runtime/lib/bionic/libc.so (__pthread_start(void*)+20) (BuildId: b3a547af3eb880c5dfd66f61a8c8b4ca)
2021-12-02 15:14:38.713 11099-11099/? A/DEBUG: #30 pc 00055a97 /apex/com.android.runtime/lib/bionic/libc.so (__start_thread+30) (BuildId: b3a547af3eb880c5dfd66f61a8c8b4ca)

How to setup devenv?

I'd like to contribute to this project, but I failed to setup devenv. Can you please provide more instructions?

how to use facesdk_eyelandmark3d API ?

Hi.
Follow your demo, I guess it should read the detect model at first, so
facesdk_readModelFromFile(ModelType::Detect, "../models/detect.bin", ImageFormat::RGB);
sdkFaces faces = facesdk_detect(data);

and then, I try to read the iris_landmark.bin model,
facesdk_readModelFromFile(ModelType::Iris, "../models/iris_landmark.bin", ImageFormat::RGB);
sdkFaces3d faces1 = facesdk_eyelandmark3d();

However, the faces1.face3d_count always equals to 0.
Anything wrong in my steps?

demo 报错

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted (core dumped)

Used models information

Hello!
Thanks for your work, it is really nice.

Can you please provide information about the models: architectures, source training framework, training data, evaluation metrics? Maybe you have a paper describing any of these?

Thanks in advance.

Issue when changing camera

I changed camera index from 0 to 1. (back camera)
and the recognitions all fail.
any potential fix to this?

It only works well for front camera frames, not back camera frames.

怎么替换自己的模型啊?

如果我用yolox训练了一个行人检测模型,转成tengine格式后,怎么替换进去?有没有相关教程啥的?

关键点

这个关键点的检测效果真心不错,很好奇怎么实现了几乎没有抖动的

please advice: TengineKit/Khadas/sample/Face/bin/Face crash with error of load plugin failed: libvxplugin.so: cannot open shared object file: No such file or directory

Hello TengineKit,

Please adivce:
TengineKit/Khadas/sample/Face/bin/Face crash with error of "load plugin failed: libvxplugin.so: cannot open shared object file: No such file or directory"

Note:
A similar library (OpenVX implementation?) exist with the Khadas VIM3 NPU SDK at:
.../aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/build/sdk/drivers_64/libovxlib.so

Appended below are the detailes of the build and run.

Build the VIM3 NPU test - ​https://github.com/OAID/TengineKit :

Reference:

  1. SSH to a Khadas VIM3 board (i.e. Amlogic A311D 4GB RAM with a 5 TOPS int8 NPU - ​​https://www.khadas.com/product-page/vim3 )
    ssh -X noveto@vim3-001
    1.1. The Khadas VIM3 board is installed with a Ubuntu 20.04 desktop and the default 4.9.x kernel:

$ uname -a
Linux vim3-001 4.9.241 #1 SMP PREEMPT Sun Dec 27 17:09:46 IST 2020 aarch64 aarch64 aarch64 GNU/Linux

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.1 LTS
Release: 20.04
Codename: focal

$ g++ --version
g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

  1. Setup dependencies
    sudo apt-get install cmake make g++ git

  2. Build the TEngine lite dependency - follow build instructions from ​https://gitee.com/OAL/Tengine/blob/tengine-lite/doc/compile.md
    cd /opt/noveto/src/FaceDetection
    ... ... ...
    3.1. After build + install, the install located is /opt/noveto/src/FaceDetection/Tengine-Lite/build/install

  3. Download sources of TengineKit
    cd /opt/noveto/src/FaceDetection
    git clone https://github.com/OAID/TengineKit && cd TengineKit

  4. Verify
    $ git log
    commit 5672e38 (HEAD -> master, origin/master, origin/HEAD)
    Author: zhangjun [email protected]
    Date: Fri Jan 15 17:46:22 2021 +0800

    Linux sample
    ... ... ...

  5. Fixup: add the expected link to the TEngine lite libraries
    cd /opt/noveto/src/FaceDetection/TengineKit/Khadas/sample/Face
    ln -s /opt/noveto/src/FaceDetection/Tengine-Lite/build/install/lib libs

  6. Build
    cd /opt/noveto/src/FaceDetection/TengineKit/Khadas/sample/Face
    bash make.sh

  7. Run
    cd /opt/noveto/src/FaceDetection/TengineKit/Khadas/sample/Face/bin
    ./Face
    load plugin failed: libvxplugin.so: cannot open shared object file: No such file or directory Load vx plugin failed.
    Segmentation fault (core dumped)
    8.1. Look for the offending code (i.e. the loading of libvxplugin.so)
    $ git blame Khadas/sample/Face/src/manager.cpp | grep libvxplugin.so
    b6cd822 (zhangjun 2021-01-04 15:23:04 +0800 19) if (load_tengine_plugin(VXDEVICE, "libvxplugin.so", "vx_plugin_init") < 0) {
    8.2. the change description is:
    $ git show b6cd822 | head
    commit b6cd822
    Author: zhangjun [email protected]
    Date: Mon Jan 4 15:23:04 2021 +0800

    update khadas

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.