xiang-wuu / ncnn-android-yolov7 Goto Github PK

I cloned this repository and I replaced the model in "app\src\main\assets" with one I trained myself on YOLOv7.
I trained it using the "yolov7.pt" weights and the results were great.
I followed the official WIKI to get the ONNX and then NCNN model: https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx

Basically I used torch.onnx._export() to create the onnx, then onnxsim to create the simplified version and then onnx2ncnn to get the NCNN version.
So I got the .bin and .param files and I renamed them "yolov7-tiny.bin" and "yolov7-tiny.param" so that I can just paste and replace the existing files in "app\src\main\assets"

I then just built the project using Android Studio, and my "local.properties" are:
sdk.dir=C:\Users\GiannisM\AppData\Local\Android\Sdk
ndk.dir=C:\Users\GiannisM\AppData\Local\Android\Sdk\ndk\25.0.8775105
cmake.dir=C:\Users\GiannisM\AppData\Local\Android\Sdk\cmake\3.22.1

The build was successful so I got the .apk file and I installed it on my phone.
The app works and I can change front and back camera as well as CPU or GPU.
I can see what the camera sees, but object detections isn't working, it's just like a camera app - nothing is being detected.

When I build it with the original "yolov7-tiny.bin" and "yolov7-tiny.param" however, it does draw bounding boxes and it shows the labels.
I should mention that when I evaluate my trained model in python, it does draw bounding boxes and labels - it's just that it doesn't work on the android app.
Any idea why that is?

Lastly, I should say that I couldn't get onnx2ncnn to work on windows so I installed and used it from WSL (Linux inside windows 11)
I found this post (https://programs.wiki/wiki/622f14b67fea9.html) that mentions things which the official WIKI doesn't mention.
It says that you should have protobuf and opencv installed before you install ncnn.
I tried it as it was said on the official WIKI without installing protobuf and opencv and no errors occurred (but like I said, when i open the app I get no detections).
I then uninstalled ncnn, installed protobuf, but the opencv installation fails. Not sure if this is the problem

Update:
I actually downloaded the pre-compiled windows binaries that include onnx2ncnn but the same things happens where I get video feed but no detections
I also managed to build and install opencv on linux (WSL) and then install ncnn afterwards and the onnx2ncnn works, but I have the exact same problem - no detections

Also, with the original yolov7-tiny.bin I get 5FPS on my "Razer Phone 2", and with my model it's 30FPS which makes me think that it really just doesn't do any forward passes to do object detection at all.

I use yolov7 models/export.py export xx.pt to xx.onnx. But use this xx.onnx not work

$ ./onnx2ncnn.exe test.onnx test.parma test.bin
Identity not supported yet!
Unknown data type 0
Unsupported Resize scales and sizes are all empty!

Can you give me some guide?
Thanks

Congratulations on the excellent work!
Unfortunately, I am unable to convert my own weights (trained with yolov7-tiny) to ncnn.
I successfully convert from pytorch (.pt) to onnx , but when trying to convert from onnxncnn I get the errors below:

EfficientNMS_TRT not supported yet!
background_class 7
box_coding 7
iou_threshold=0.5
max_output_boxes=100
plugin_version=1
score_activation=0
score_threshold=0.15

I'm using the tools provided at https://convertmodel.com/#input=onnx&output=onnx and at https://github.com/Tencent/ncnn/wiki/use-ncnn-with-pytorch-or-onnx#simplify- onnx-model

Could you help me by letting me know step by step how you converted your weights from yolov7-tiny.pt to yolov7-tiny.ncnn? This is very important to me.

Hi, I cloned this repo and tried to build the app but I'm getting clang++: error: unknown argument: '-static-openmp error, the dependencies I use for builing are these:

sdk.dir=/Users/martinpavlik/Library/Android/sdk
ndk.dir=/Users/martinpavlik/Library/Android/sdk/ndk/20.1.5948944
cmake.dir=/Users/martinpavlik/Library/Android/sdk/cmake/3.10.2.4988404

Build command failed.
Error while executing process /Users/martinpavlik/Library/Android/sdk/cmake/3.10.2.4988404/bin/ninja with arguments {-C /Users/martinpavlik/work/ncnn-android-yolov7/app/.cxx/cmake/debug/armeabi-v7a ncnnyolov7}
ninja: Entering directory `/Users/martinpavlik/work/ncnn-android-yolov7/app/.cxx/cmake/debug/armeabi-v7a'
[0/1] Re-running CMake...
-- Found OpenMP_C: -fopenmp=libomp  
-- Found OpenMP_CXX: -fopenmp=libomp  
-- Found OpenMP: TRUE   
-- Found OpenMP_C: -fopenmp=libomp  
-- Found OpenMP_CXX: -fopenmp=libomp  
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/martinpavlik/work/ncnn-android-yolov7/app/.cxx/cmake/debug/armeabi-v7a
[1/1] Linking CXX shared library /Users/martinpavlik/work/ncnn-android-yolov7/app/build/intermediates/cmake/debug/obj/armeabi-v7a/libncnnyolov7.so
FAILED: /Users/martinpavlik/work/ncnn-android-yolov7/app/build/intermediates/cmake/debug/obj/armeabi-v7a/libncnnyolov7.so 
: && /Users/martinpavlik/Library/Android/sdk/ndk/20.1.5948944/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ --target=armv7-none-linux-androideabi24 --gcc-toolchain=/Users/martinpavlik/Library/Android/sdk/ndk/20.1.5948944/toolchains/llvm/prebuilt/darwin-x86_64 --sysroot=/Users/martinpavlik/Library/Android/sdk/ndk/20.1.5948944/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -fno-addrsig -march=armv7-a -mthumb -Wa,--noexecstack -Wformat -Werror=format-security   -O0 -fno-limit-debug-info  -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id -Wl,--warn-shared-textrel -Wl,--fatal-warnings -Wl,--exclude-libs,libunwind.a -Wl,--no-undefined -Qunused-arguments -Wl,-z,noexecstack -shared -Wl,-soname,libncnnyolov7.so -o /Users/martinpavlik/work/ncnn-android-yolov7/app/build/intermediates/cmake/debug/obj/armeabi-v7a/libncnnyolov7.so CMakeFiles/ncnnyolov7.dir/yoloncnn.cpp.o CMakeFiles/ncnnyolov7.dir/yolo.cpp.o CMakeFiles/ncnnyolov7.dir/ndkcamera.cpp.o  /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/ncnn/armeabi-v7a/lib/libncnn.a /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/opencv/sdk/native/staticlibs/armeabi-v7a/libopencv_core.a /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/opencv/sdk/native/staticlibs/armeabi-v7a/libopencv_imgproc.a -lcamera2ndk -lmediandk -Wl,-wrap,__kmp_affinity_determine_capable /Users/martinpavlik/Library/Android/sdk/ndk/20.1.5948944/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/arm-linux-androideabi/24/libvulkan.so /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/ncnn/armeabi-v7a/lib/libglslang.a /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/ncnn/armeabi-v7a/lib/libSPIRV.a /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/ncnn/armeabi-v7a/lib/libMachineIndependent.a /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/ncnn/armeabi-v7a/lib/libOGLCompiler.a /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/ncnn/armeabi-v7a/lib/libOSDependent.a /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/ncnn/armeabi-v7a/lib/libGenericCodeGen.a -landroid -ljnigraphics /Users/martinpavlik/work/ncnn-android-yolov7/app/src/main/jni/opencv/sdk/native/staticlibs/armeabi-v7a/libopencv_core.a -fopenmp -static-openmp -ldl -lm -llog -latomic -lm && :
clang++: error: unknown argument: '-static-openmp'
ninja: build stopped: subcommand failed.

What is wrong please? Thanks

It is currently using front selfie camera by default.
How can I switch it to use the rear facing camera instead?

Hi, is there any way to increase the fps?
The fps seems to be a little bit low...
Thanks!

Hi, thank you for the awesome demo！
I have a few questions about using cutom weights

1. There are some errors when exporting onnx to ncnn : Unsupported slice axes !, and the result can't be used in detection
2. Can I use models other than yolov7-tiny? If so , do I need to change any code besides replacing the assets?
   Thanks!

Thank you for your great work! It is very useful.
But when I use onnx2ncnn.exe in ncnn to transfer yolov7.onnx into bin and param, I got these errors:

Unsupported slice axes !
ScatterND not supported yet!
Unsupported slice axes !
Expand not supported yet!
ScatterND not supported yet!
Unsupported slice axes !
ScatterND not supported yet!
Unsupported slice axes !
Expand not supported yet!
ScatterND not supported yet!
Unsupported slice axes !
ScatterND not supported yet!
Unsupported slice axes !
Expand not supported yet!
ScatterND not supported yet!

And the created bin and param can not work in my android phone by your code. While when I can use the default bin and param in your code, it works.

I use the way you mentioned in #2 to transfer yolov7.pt into yolov7.onnx.

Thank you for your answer!

The following error occurs when I building. NDK version is 22.1.7171670
Error while executing process E:\download\Android\Sdk\cmake\3.10.2.4988404\bin\ninja.exe with arguments {-C D:\face\ncnn-android-yolov7-master\app\.cxx\cmake\debug\arm64-v8a ncnnyolov7} ninja: Entering directoryD:\face\ncnn-android-yolov7-master\app.cxx\cmake\debug\arm64-v8a'
[1/4] Building CXX object CMakeFiles/ncnnyolov7.dir/yoloncnn.cpp.o
[2/4] Building CXX object CMakeFiles/ncnnyolov7.dir/ndkcamera.cpp.o
[3/4] Building CXX object CMakeFiles/ncnnyolov7.dir/yolo.cpp.o
[4/4] Linking CXX shared library D:\face\ncnn-android-yolov7-master\app\build\intermediates\cmake\debug\obj\arm64-v8a\libncnnyolov7.so
FAILED: D:/face/ncnn-android-yolov7-master/app/build/intermediates/cmake/debug/obj/arm64-v8a/libncnnyolov7.so
cmd.exe /C "cd . && E:\download\Android\Sdk\ndk\22.1.7171670\toolchains\llvm\prebuilt\windows-x86_64\bin\clang++.exe --target=aarch64-none-linux-android24 --gcc-toolchain=E:/download/Android/Sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/windows-x86_64 --sysroot=E:/download/Android/Sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/windows-x86_64/sysroot -fPIC -g -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -O0 -fno-limit-debug-info -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libgcc_real.a -Wl,--exclude-libs,libatomic.a -static-libstdc++ -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--fatal-warnings -Wl,--no-undefined -Qunused-arguments -shared -Wl,-soname,libncnnyolov7.so -o D:\face\ncnn-android-yolov7-master\app\build\intermediates\cmake\debug\obj\arm64-v8a\libncnnyolov7.so CMakeFiles/ncnnyolov7.dir/yoloncnn.cpp.o CMakeFiles/ncnnyolov7.dir/yolo.cpp.o CMakeFiles/ncnnyolov7.dir/ndkcamera.cpp.o D:/face/ncnn-android-yolov7-master/app/src/main/jni/ncnn-20220729-android/arm64-v8a/lib/libncnn.a D:/face/ncnn-android-yolov7-master/app/src/main/jni/opencv-mobile-4.5.4-android/sdk/native/staticlibs/arm64-v8a/libopencv_core.a D:/face/ncnn-android-yolov7-master/app/src/main/jni/opencv-mobile-4.5.4-android/sdk/native/staticlibs/arm64-v8a/libopencv_imgproc.a -lcamera2ndk -lmediandk -fopenmp -static-openmp -landroid -ljnigraphics D:/face/ncnn-android-yolov7-master/app/src/main/jni/opencv-mobile-4.5.4-android/sdk/native/staticlibs/arm64-v8a/libopencv_core.a -ldl -lm -llog -latomic -lm && cd ."
ld: error: undefined symbol: __aarch64_ldadd4_acq_rel

referenced by mat.cpp
mat.cpp.o:(ncnn::Mat::clone(ncnn::Allocator*) const) in archive D:/face/ncnn-android-yolov7-master/app/src/main/jni/ncnn-20220729-android/arm64-v8a/lib/libncnn.a
referenced by mat.cpp
mat.cpp.o:(ncnn::Mat::clone(ncnn::Allocator*) const) in archive D:/face/ncnn-android-yolov7-master/app/src/main/jni/ncnn-20220729-android/arm64-v8a/lib/libncnn.a
referenced by mat.cpp
mat.cpp.o:(ncnn::Mat::create(int, unsigned long, int, ncnn::Allocator*)) in archive D:/face/ncnn-android-yolov7-master/app/src/main/jni/ncnn-20220729-android/arm64-v8a/lib/libncnn.a
referenced 2260 more times
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
`

I have successfully run yolov7-tiny.bin but don't know how to convert yolov7 & yolov7-w6-pose.pt to run on android application

你好，哥，我是做安卓开发的，没做过音视频底层开发那块，所以我没怎么学过C++，现在需要用到人工智能，但大多的框架是针对yolov5目标检测的，少有物体分类。

The project requires JDK11 and android-ndk-r21e.
You need to download the NDK from github manually.

By the way, this is the version that doesn't need downgrading (2023/05/22) :
https://drive.google.com/file/d/12IyuXRKKGG2Zh_PBLE6dPFCeEGQYuiNp/view?usp=sharing
New a Native C++ project named "MyJNI" and replace the app/scr/main.

Hi, thanks for the great implementation.

I've tried running on my android device, but been facing cmake related errors.

  Error while executing process /Users/gea_hs/Library/Android/sdk/cmake/3.10.2.4988404/bin/ninja with arguments {-C ~/Downloads/ncnn-android-yolov7-master/app/.cxx/cmake/debug/arm64-v8a ncnnyolov7}
ninja: Entering directory `/Users/gea_hs/Downloads/ncnn-android-yolov7-master/app/.cxx/cmake/debug/arm64-v8a'
  [1/4] Building CXX object CMakeFiles/ncnnyolov7.dir/yoloncnn.cpp.o
  [2/4] Building CXX object CMakeFiles/ncnnyolov7.dir/ndkcamera.cpp.o
  [3/4] Building CXX object CMakeFiles/ncnnyolov7.dir/yolo.cpp.o
  [4/4] Linking CXX shared library ~/Downloads/ncnn-android-yolov7-master/app/build/intermediates/cmake/debug/obj/arm64-v8a/libncnnyolov7.so

I followed step 1 and step 2, but is there any other pre-requisite?

Thanks,

Hi, thank you for the awesome demo！
I have a few questions about using cutom weights
When I used the FP16 model, there was no change in speed.
Can I use fp16 model? If so, where should I change the code.
Thanks!

ninja: error: '/usr/local/lib/android/sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/14.0.7/lib/linux/arm/libomp.so', needed by 'D:/studioworkspace/ncnn-android-yolov7-master/app/build/intermediates/cmake/debug/obj/armeabi-v7a/libncnnyolov7.so', missing and no known rule to make it

at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:67)
at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:41)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:106)

老师按步骤配置编译一直报上述错误,以下为CMakeLists.txt的配置
project(ncnnyolov7)

cmake_minimum_required(VERSION 3.10)

set(OpenCV_DIR ${CMAKE_SOURCE_DIR}/opencv-mobile-4.8.0-android/sdk/native/jni)
find_package(OpenCV REQUIRED)

set(ncnn_DIR ${CMAKE_SOURCE_DIR}/ncnn-20230517-android-vulkan/${ANDROID_ABI}/lib/cmake/ncnn)
find_package(ncnn REQUIRED)

add_library(ncnnyolov7 SHARED yoloncnn.cpp yolo.cpp ndkcamera.cpp)

target_link_libraries(ncnnyolov7 ncnn ${OpenCV_LIBS} camera2ndk mediandk)

may I ask how to solve this crush problem? Or could any advice to learn to solve this problem? Sorry I new on andorid deploy.

Hi, thanks for the sources.

I am now looking into the app sources and deploying it on my device to test.
I have met the problem in changing the camera resolution.
Can I ask you how to change the camera resolution?

Thanks,

The following error occurred when switching to the model trained by myself:
Fatal signal 11 (SIGSEGV), code 1 (SEGV-MAPERR), fault addr 0xf in tid 22018 (center. ncnnyolov7), pid 22018 (center. ncnnyolov7)

I added models from YOLOv6, but it is not working, probably models are using different formats?
I downloaded YOLOv6 models from:
https://github.com/xiang-wuu/ncnn-android-yolov6/tree/main/app/src/main/assets

Can you add image direct detect interface ? Thanks