Giter Club home page Giter Club logo

vnn's Introduction

Logo

简介

VNN是由欢聚集团(Joyy Inc.)推出的高性能、轻量级神经网络部署框架。目前已为HagoVOOVFlyCloudVFly马克水印相机等App提供20余种AI能力的支持,覆盖直播、短视频、视频编辑等泛娱乐场景和工程场景

特性

  • 免费:无授权控制,无广告植入
  • 安全:零权限需求
  • 跨平台一致设计和体验:良好支持Andorid/iOS/Windows/MacOS(Intel & Apple Chip)/Linux系统平台
  • 轻盈:全部功能SDK打包体积 Andorid 2.3M(arm64), iOS 2.0M(arm64)
  • 易用:API风格简洁、统一

支持能力

ability


人脸关键点

face_Img face_keypoints face_Img face278
原图 人脸关键点 美颜 动态效果

指标对比

face104compare face278compare

各功能效果展示

disney_example 3dgame_example comic_example
迪士尼人脸风格化 3D游戏人脸风格化 油画图像风格化
portrait_segment_example animal_segment_example sky_segment_example
人像分割 动物分割 天空分割
clothes_segment_example animal_segment_example face_reenactment_example
衣物分割(换色) 头发分割(换色) 人脸表情驱动

Demo体验

平台 Andorid iOS Windows MacOS Linux
apk / 工程 工程 exe / 工程 工程 工程

集成指引与SDK文档


Acknowledgement

参考了以下项目:


更新计划

  • 发布模型转换工具链以及模型调用接口
  • 发布libs核心源代码

License

MIT


注明

如果有商用或使用问题可以随时联系我们:
[email protected], [email protected], [email protected], [email protected]

vnn's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vnn's Issues

卡顿

在小米8SE安卓设备上,3D、卡通等效果,明显感觉卡顿

风格化效果求助

1、使用迪士尼和3Dgame的效果时会裁掉一部分头发的区域,怎么扩展处理的区域?如果使用人脸mask,整体会不太协调,因为只处理的人脸的部分,头发、脖子等未处理。
2、怎么提升输出效果的清晰度?

个别pc机器上的人像分割崩溃

崩溃堆栈如下:
vnn_core!venus::runtime::Serializable::~Serializable+0x19ca
vnn_core!venus::runtime::Conv2dInfo::VisitAttr+0x210
vnn_core!venus::runtime::Serializable::ToFile+0x1927
vnn_core!venus::runtime::VNModelInfo::VisitAttr+0x1480
vnn_core!venus::runtime::Serializable::ToFile+0x1927
vnn_core!venus::runtime::VNModelInfoArr::VisitAttr+0x2e2
vnn_core!venus::runtime::Serializable::ToFile+0x1927
vnn_core!venus::runtime::VNNModel::VisitAttr+0xa2
vnn_core!venus::runtime::Serializable::FromFile+0x4ac
vnn_core!venus::utility::load_model+0x244
vnn_general!venus::runtime::AlignedPool::default constructor closure'+0x153b vnn_general!venus::runtime::AlignedPool::default constructor closure'+0x13f4
vnn_general!VNN_Create_General+0x23c

disney和3dgame

disney和3dgame,对输入的图片或者视频帧,有最小尺寸限制吗?

Android demo is giving issue at compiling

macbook air m1

Android Studio Flamingo | 2022.2.1 Patch 2
ndk: 21.1.635245
cmake:3.6.4111459

Execution failed for task ':vnnlib:generateJsonModelDebug'.

Build command failed.

Error while executing process .../cmake/3.6.4111459/bin/cmake with arguments {-H/.../VNN-master/demos/Android/vnn_android_demo/vnnlib

Android demo is giving issue at compiling

/src/main/cpp/vnnjni.cpp:59: undefined reference to `VNN_Destroy_Face'

/src/main/cpp/vnnjni.cpp:1197: undefined reference to `VNN_Destroy_ObjCount'

./../src/main/cpp/vnnjni.cpp:1313: undefined reference to `VNN_Destroy_DocRect'

Can I know a specific version of the CMake tool and NDK version which is used at the time of compiling demos

Nice!! 期待开放GPU接口~

public static class VNN_Image {
public long ori_fmt; /* orientation format enum of img*/
public int pix_fmt; /* pixel format enum of img /
public int mode_fmt; /
mode format enum of detection, can use video/picture/default /
public int width; //image width
public int height; //image height
public int channels; //image channel
public byte[] data; //cpu data
public float[] rect;
public int[] texture; /
if is cpu-backend, texture is NULL. But if is gpu-backend, texture may be a VNN_Texture(android|ios) or a CVPixelbuffer(ios). */
}

native崩溃

你好,我在使用vnn进行facemask检测时遇到了一些native崩溃,这是我的检测代码:
``
public static Bitmap vnnFaceMask(Bitmap bitmap, boolean recycle, float[] rect) {
Bitmap result = Bitmap.createBitmap(128, 128, Bitmap.Config.ARGB_8888);
result.eraseColor(Color.BLACK);
try {
VNNHelper vnnHelper = new VNNHelper(App.appContext);
vnnHelper.createModels(VNNHelper.VNN_EFFECT_MODE.VNN_FACE_MASK);
VNN.VNN_Image inputImage = new VNN.VNN_Image();
inputImage.width = bitmap.getWidth();
inputImage.height = bitmap.getHeight();
inputImage.data = BitmapUtil.getRGBAFromBitmap(bitmap);
inputImage.ori_fmt = VNN.VNN_OrientationFormat.VNN_ORIENT_FMT_DEFAULT;
inputImage.pix_fmt = VNN.VNN_PixelFormat.VNN_PIX_FMT_RGBA8888;
inputImage.mode_fmt = VNN.VNN_MODE_FMT.VNN_MODE_FMT_PICTURE;
vnnHelper.apply(VNNHelper.VNN_EFFECT_MODE.VNN_FACE_MASK, inputImage, null);
VNN.VNN_ImageArr imageArr = vnnHelper.imageArr;
if (recycle) {
BitmapUtil.recycleBitmap(bitmap);
}
if (imageArr.imgsNum == 0) {
return result;
}
int width = imageArr.imgsArr[0].width;
int height = imageArr.imgsArr[0].height;
int[] pix = new int[width * height];
for (int j = 0; j < pix.length; j++) {
pix[j] = imageArr.imgsArr[0].data[j] == 0 ? Color.BLACK : Color.WHITE;
}
result.setPixels(pix, 0, width, 0, 0, width, height);
if (rect != null) {
System.arraycopy(imageArr.imgsArr[0].rect, 0, rect, 0, 4);
}
vnnHelper.destroyVNN(VNNHelper.VNN_EFFECT_MODE.VNN_FACE_MASK);
} catch (Error e) {
e.printStackTrace();
}
return result;
}

测试环境和崩溃信息如下:
1、
测试机型:oppo A37m
崩溃日志:
E/<<<Native_Capturer>>>: handle signal: signal=11 sigCode=1 errno=0
E/xunwind: #00 pc 00000000000171a8 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #1 pc 000000000000afc0 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #2 pc 0000000000007b44 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #3 pc 0000000000007974 /data/app/应用包名-2/lib/arm64/libvnn_face.so
E/xunwind: #4 pc 0000000000005620 /data/app/应用包名-2/lib/arm64/libvnn_face.so (VNN_Apply_Face_CPU+144)
E/xunwind: #5 pc 0000000000003abc /data/app/应用包名-2/lib/arm64/libvnnjni.so (Java_com_duowan_vnnlib_VNN_applyFaceCpu+580)
E/xunwind: #6 pc 000000000217dbd4 /data/dalvik-cache/arm64/data@app@应用包名-2@[email protected]
E/NativeCapturer: crash:
####
handleThread=Thread-3177
crashThread=GLHandlerThread
####

2、这个问题比较奇怪,在某次编译出来的apk包出现这个问题,在未修改检测代码的情况下,重新编译出来的apk未复现这个问题
测试机型:小米11、红米6A
崩溃日志:
2022-02-14 15:20:47.875 30880-32060/? A/libc: Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 32060 (GLHandlerThread), pid 30880 (cordion.prettyo)
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Build fingerprint: 'Xiaomi/venus/venus:11/RKQ1.200928.002/V12.5.13.0.RKBCNXM:user/release-keys'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Revision: '0'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: ABI: 'arm64'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Timestamp: 2022-02-14 15:20:48+0800
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: pid: 30880, tid: 32060, name: GLHandlerThread >>> 应用包名 <<<
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: uid: 14387
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: Abort message: 'JNI DETECTED ERROR IN APPLICATION: fid == null
in call to GetLongField
from int com.duowan.vnnlib.VNN.applyFaceCpu(int, com.duowan.vnnlib.VNN$i, com.duowan.vnnlib.VNN$f)'
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x0 0000000000000000 x1 0000000000007d3c x2 0000000000000006 x3 0000007d0ce792d0
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x4 000024393f4f282a x5 000024393f4f282a x6 000024393f4f282a x7 2a284f3f3924ffff
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x8 00000000000000f0 x9 a4bd9e266b09fda5 x10 0000000000000000 x11 ffffffc0fffffbdf
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x12 0000000000000001 x13 00000000000000e2 x14 ffff000000000000 x15 ffffffffffffffff
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x16 0000007d9fbd6948 x17 0000007d9fbb5350 x18 0000000000000001 x19 00000000000078a0
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x20 0000000000007d3c x21 00000000ffffffff x22 0000000000000058 x23 0000000000000058
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x24 0000007d1bbd5157 x25 0000000000000001 x26 0000007d1bbec171 x27 0000007d1c1f2000
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: x28 b400007d1c6d1c00 x29 0000007d0ce79350
2022-02-14 15:20:48.016 32149-32149/? A/DEBUG: lr 0000007d9fb68a60 sp 0000007d0ce792b0 pc 0000007d9fb68a8c pst 0000000000001000
2022-02-14 15:20:48.018 32149-32149/? A/DEBUG: backtrace:
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: NOTE: Function names and BuildId information is missing for some frames due
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: NOTE: to unreadable libraries. For unwinds of apps, only shared libraries
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: NOTE: found under the lib/ directory are readable.
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #00 pc 0000000000089a8c /apex/com.android.runtime/lib64/bionic/libc.so (abort+164) (BuildId: 5f57d25b37c043ed36c0e4147dcc8b3f)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #1 pc 000000000055de88 /apex/com.android.art/lib64/libart.so (art::Runtime::Abort(char const*)+2308) (BuildId: 5b103c304a50c13c7fbeaacc0e0df496)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #2 pc 0000000000013978 /system/lib64/libbase.so (android::base::SetAborter(std::__1::function<void (char const*)>&&)::$_3::__invoke(char const*)+76) (BuildId: 7f81cc97d60164bfae3b52a2cd822f20)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #3 pc 0000000000012fa4 /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+320) (BuildId: 7f81cc97d60164bfae3b52a2cd822f20)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #4 pc 0000000000386344 /apex/com.android.art/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+2572) (BuildId: 5b103c304a50c13c7fbeaacc0e0df496)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #5 pc 00000000003becf4 /apex/com.android.art/lib64/libart.so (art::JNI::GetLongField(_JNIEnv*, _jobject*, _jfieldID*)+1624) (BuildId: 5b103c304a50c13c7fbeaacc0e0df496)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #6 pc 0000000000003904 /data/app/~~tkynBF6I6wYFbvaZ834Qpg==/应用包名-Dgei18UFSwJBh-34YYpIig==/lib/arm64/libvnnjni.so (Java_com_duowan_vnnlib_VNN_applyFaceCpu+140) (BuildId: 7ad79b2d6a5b50e2a8bba41a5833ceae7ecdcb3d)
2022-02-14 15:20:48.019 32149-32149/? A/DEBUG: #7 pc 0000000000050360 /data/app/~~tkynBF6I6wYFbvaZ834Qpg==/应用包名-Dgei18UFSwJBh-34YYpIig==/oat/arm64/base.odex (deleted)
2022-02-14 15:20:48.024 1676-2446/? E/NsdService: id 364996 for SERVICE_GET_ADDR_SUCCESS has no client mapping
2022-02-14 15:20:48.024 1676-2446/? E/NsdService: Unhandled { when=0 what=393242 obj=com.android.server.NsdService$NativeEvent@a2708df target=com.android.internal.util.StateMachine$SmHandler }

由于没有开源代码,我无法定位和修改问题,希望能够解决,谢谢!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.