sthphoenix / insightface-rest Goto Github PK

View Code? Open in Web Editor NEW

464.0 19.0 109.0 4.49 MB

InsightFace REST API for easy deployment of face recognition services with TensorRT in Docker.

License: Apache License 2.0

Shell 2.05% Python 97.95%

retinaface fastapi tensorrt tensorrt-conversion arcface face-recognition face-detection onnx gpu docker

insightface-rest's People

Contributors

Stargazers

Watchers

Forkers

danigunawan dualven wuqiangch alicera xxxpsyduck davorjordacevic kzn projektosmium arcticroman hoangtienduc lbtanh deaffella jackhouchina 10183308 diegosiqueir4 benjamesbabala rena-ganba narutolixiaodong wjj962464 thanx228 sinianyutian corner4world yiouyou file-campuran alighofrani95 vuminhduc97 hitzht ngocson1804 gulldan atomeyang connortran216 samsu2018 u0251077 rsingh2083 victorgirleanu amhoho cony2021 tuanlase02874 zgd716 jsompis djord97 a7ypically netgsm oleh-ozimok placeboooo 7aughing felixdollack iamsvp94 bogatykhnatalia sominesawa z3r0bro goga1992 tsyjwct nrobin dfqytcom anhlbt vladik7 hanxm715 leruge iafydsttta liuzhiliangpc lijie2160 lschaupp internnos geniusfoever yuxianmin caibao413672274 park-kumin wiw100 sumitlakra1992 kradonneoh fanhuafeng anshsarkar fengmy webclinic017 techthiyanes chuanbei888 jie0001 ouriso diningsystem mobiusml donghoang93 jinghengkhoo quyen88 eng-yasmin275 coolhuasen ruhyadi glamy2008 shnulaa kh4k1m palatinevision hamzaziizzz rohitech reynold97 sihabsahariar bingqingsuimeng tmq902005 anhngml aifaq thlo7777

insightface-rest's Issues

ValueError: operands could not be broadcast together with shapes (288,) (338,)

import os
import json
import base64
import requests
from glob import glob


def file2base64(path):
    with open(path, mode='rb') as fl:
        encoded = base64.b64encode(fl.read()).decode('ascii')
        return encoded


def extract_vecs(ims, max_size=[200, 200]):
    target = [file2base64(im) for im in ims]
    req = {"images": {"data": target}, "max_size": max_size}
    resp = requests.post('http://localhost:18081/extract', json=req)
    data = resp.json()
    return data


images_path = '/src/api_trt/test_images/*'
images = glob(images_path)
data = extract_vecs(images)

When I run docker logs container_id, I get error

  File "/app/./modules/model_zoo/detectors/scrfd.py", line 39, in distance2bbox
    x1 = points[:, 0] - distance[:, 0]
ValueError: operands could not be broadcast together with shapes (288,) (338,)

Package 'libturbojpeg' has no installation candidate

Hi, I was running bash deploy_cpu.sh and got this error. Do you have any suggestion? Thanks.

Sending build context to Docker daemon  3.729MB
Step 1/14 : FROM python:3.8-slim
3.8-slim: Pulling from library/python
69692152171a: Pull complete
66a3c154490a: Pull complete
3e35bdfb65b2: Pull complete
f2c4c4355073: Pull complete
21bb8d414880: Pull complete
Digest: sha256:1156cbb1f6a7660dcce3e2f3906a149427fbee71aea0b49953bccf0cc7a3bcaa
Status: Downloaded newer image for python:3.8-slim
 ---> 40663d3c7bf7
Step 2/14 : ENV LC_ALL=C.UTF-8
 ---> Running in d6dfdc1243c9
Removing intermediate container d6dfdc1243c9
 ---> 3b1f9ca09012
Step 3/14 : ENV LANG=C.UTF-8
 ---> Running in 880da5716b1a
Removing intermediate container 880da5716b1a
 ---> 4575dcd4f5fe
Step 4/14 : RUN apt-get update && apt-get install -y libgl1-mesa-glx protobuf-compiler libprotoc-dev libb64-0d libglib2.0-0     libgomp1 gcc curl  libturbojpeg && rm -rf /var/lib/apt/lists/*
 ---> Running in c024e868d09e
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:2 http://security.debian.org/debian-security buster/updates/main amd64 Packages [289 kB]
Get:3 http://deb.debian.org/debian buster InRelease [121 kB]
Get:4 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [10.9 kB]
Fetched 8445 kB in 5s (1589 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
Package libturbojpeg is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  libturbojpeg0

E: Package 'libturbojpeg' has no installation candidate
The command '/bin/sh -c apt-get update && apt-get install -y libgl1-mesa-glx protobuf-compiler libprotoc-dev libb64-0d libglib2.0-0     libgomp1 gcc curl  libturbojpeg && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100
Starting 1 workers on CPU
Error response from daemon: No such container: insightface-rest-cpu
Error: No such container: insightface-rest-cpu
--- Starting container insightface-rest-cpu with CPU at port 18081
Unable to find image 'insightface-rest:v0.5.9.9-cpu' locally
docker: Error response from daemon: pull access denied for insightface-rest, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'

Issue running pipelinetest.py

hi @SthPhoenix :

I followed all the steps , starting from creating docker to doing all the installation . ( Model download and conversion for both detection and recognition )

Upon completion of that while I am trying to run pipeline_tester.py file inside container I am getting this error :
Can you please help me with this?

Traceback (most recent call last):
File "pipeline_tester.py", line 18, in
faces = model.get(image)
File "/app/modules/face_model.py", line 204, in get
ret = [e for e in self.embed_faces(ret)]
File "/app/modules/face_model.py", line 204, in
ret = [e for e in self.embed_faces(ret)]
File "/app/modules/face_model.py", line 132, in embed_faces
embeddings = self.rec_model.get_embedding(crops)

Error: No such container: insightface-rest-gpu0-trt

./deploy_trt.sh gives below error

Error: No such container: insightface-rest-gpu0-trt

why insightFace-Rest only get fewer faces when deploying on trt with default parameters compared with the result showing on readme?

Model conversion error

root@84a3a2a7c8f3:/app# python3 prepare_models.py 
mxnet version: 1.6.0
onnx version: 1.7.0
[12:45:13] INFO - Preparing 'arcface_r100_v1' model...
Downloading model: arcface_r100_v1...
Model file is not found. Downloading.
Downloading /models/mxnet/arcface_r100_v1.zip from http://insightface.ai/files/models/arcface_r100_v1.zip...
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████| 237710/237710 [03:47<00:00, 1046.77KB/s]
[12:49:02] INFO - Creating intermediate copy of source model...
[12:49:03] INFO - Applying RetinaFace specific fixes to input MXNet model before conversion...
[12:49:03] INFO - Exporting to ONNX...
[12:49:03] INFO - Converting json and weight file to sym and params
[12:49:03] src/nnvm/legacy_json_util.cc:209: Loading symbol saved by previous version v1.0.0. Attempting to upgrade...
[12:49:03] src/nnvm/legacy_json_util.cc:217: Symbol successfully upgraded!
[12:49:11] INFO - Input shape of the model [(1, 3, 112, 112)] 
[12:49:11] INFO - Exported ONNX file /models/onnx/arcface_r100_v1/arcface_r100_v1.onnx saved to disk
[12:49:11] INFO - Applying ArcFace specific fixes to output ONNX
[12:49:14] INFO - Removing initializer from inputs in ONNX model...
[12:49:14] INFO - Removing intermediate *.symbol and *.params
[12:49:14] INFO - Building TRT engine for arcface_r100_v1...
Traceback (most recent call last):
  File "prepare_models.py", line 45, in <module>
    prepare_models()
  File "prepare_models.py", line 40, in prepare_models
    prepare_backend(model_name=model, backend_name=backend_name, im_size=max_size, force_fp16=force_fp16,
  File "/app/modules/model_zoo/getter.py", line 127, in prepare_backend
    convert_onnx(temp_onnx_model,
TypeError: 'NoneType' object is not callable

Error while trying to hit /extract api on test image.

I am using deploy_cpu.sh to run a container on my local machine. The container is up and running. When I try using /extract endpoint on a test image using the example provided, I get a null response.

Request:

{
 "images": {
   "data": [
     "string"
   ],
   "urls": [
     "test_images/Stallone.jpg"
   ]
 },
 "max_size": [
   640,
   640
 ],
 "threshold": 0.6,
 "embed_only": false,
 "return_face_data": false,
 "return_landmarks": false,
 "extract_embedding": true,
 "extract_ga": true,
 "limit_faces": 0,
 "min_face_size": 0,
 "verbose_timings": true,
 "api_ver": "1"
}

Error

Traceback (most recent call last):
  File "/app/./modules/processing.py", line 235, in embed
    faces = await self.model.get(image, max_size=max_size, threshold=threshold,
  File "/app/./modules/face_model.py", line 220, in get
    faces = [e for e in self.process_faces(faces, extract_embedding=extract_embedding,
  File "/app/./modules/face_model.py", line 220, in <listcomp>
    faces = [e for e in self.process_faces(faces, extract_embedding=extract_embedding,
  File "/app/./modules/face_model.py", line 138, in process_faces
    ga = self.ga_model.get(crops)
  File "/app/./modules/model_zoo/exec_backends/onnxrt_backend.py", line 51, in get
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
cv2.error: OpenCV(4.5.2) /tmp/pip-req-build-eirhwqtr/opencv/modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<3, 4>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = cv::impl::<unnamed>::NONE; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 112

performance drop

hi, thank you for sharing your work.

I just checked that there might be a drop in the accuracy performance after testing it on LFW, where it can only achieve around 90% after conversion to onnx. Also the recall is now around 86%

Triton inference

Hello,

I'm trying to use Triton inf server (tritonserver:21.04-py3) with trt plan from your project. Model - retinaface

Your TRT backend works perfect.
docker run -p 18081:18080 -d --gpus 0 -e LOG_LEVEL=INFO -e PYTHONUNBUFFERED=0 -e NUM_WORKERS=1 -e INFERENCE_BACKEND=trt -e FORCE_FP16=True -e DET_NAME=retinaface_r50_v1 -e DET_THRESH=0.6 -e REC_NAME=glint360k_r100FC_1.0 -e REC_IGNORE=False -e REC_BATCH_SIZE=64 -e GA_NAME=genderage_v1 -e GA_IGNORE=False -e KEEP_ALL=True -e MAX_SIZE=1024,780 -e DEF_RETURN_FACE_DATA=True -e DEF_EXTRACT_EMBEDDING=True -e DEF_EXTRACT_GA=True -e DEF_API_VER='1'

But if I try to transfer your processing from
https://github.com/SthPhoenix/InsightFace-REST/blob/master/src/api_trt/modules/model_zoo/detectors/retinaface.py#L268
like postprocessing Triton's result.

dw and dh just empty list for stride16

Would you recommend something?

Triton conf
triton_model_config.zip

Jupyter notebook
triton_test.zip

Inference Script for Converted Tensorrt Model

@SthPhoenix Thanks for nice work, I have converted the arcface model to tensorrt using your repository, can you provide the inferences script on input image for the converted tensorrt model. Thanks

What's the different between retinaface_mnet025 V1 and V2?

Hi,
I only see one model on InsightFace github. I'm curious about the model on InsightFace is which version and what's the different between V1 and V2.

why did you prefer to use cython than numba gpu?

Discussion about scrfd running tensorrt

Hi. This's not related to this codebase yet but we may have the same concern so I post it here. I'm using onnx with tensorrt backend trying to do inference with model scrfd_10g_bnkps. The problem comes when output from tensorrt is different with onnx backend. In addition, anchor_centers and bbox_preds do not have matched shapes after predicting with tensorrt. Detail below:

onnx_runtime:
anchor_centers: (12800, 2)
bbox_preds: (12800, 4)
anchor_centers: (3200, 2)
bbox_preds: (3200, 4)
anchor_centers: (800, 2)
bbox_preds: (800, 4)

tensorrt:
anchor_centers: (12800, 2)
bbox_preds: (3200, 1)
anchor_centers: (3200, 2)
bbox_preds: (3200, 4)
anchor_centers: (800, 2)
bbox_preds: (3200, 10)

I hope you can take your time looking into this.

scrfd_10g_bnkps detection model cannot detect lower resolution images such as 300*300 unless given lower threshold such as 0.2 or 0.3

MAX_SIZE="640,640"
DET_THRESH=0.6 --> no faces detected
DET_THRESH=0.5 --> no faces detected
DET_THRESH=0.4 --> no faces detected
DET_THRESH=0.3 --> find faces

Convert the alignment model

@SthPhoenix Hi ,I used the code to convert the face alignment mode. https://github.com/deepinsight/insightface/tree/master/alignment/coordinateReg

but the code of checker.check_graph(graph) get the bellow error info:
onnx.onnx_cpp2py_export.checker.ValidationError: Nodes in a graph must be topologically sorted, however input 'reshape4' of node: input: "conv_1_relu_gamma" input: "reshape4" output: "slope4" name: "slope4" op_type: "Reshape" is not output of any previous nodes.

About retinaface dynamic input size

Hi @SthPhoenix,
Thank you for this repo, I've finally got proper conversion (tried with original mxnet to onnx converter and different other methods from github) with good performance. But one question regarding the conversion of original Retinaface model is how to set dynamic input size like in their example? I mean, the original model accepts different sizes and gives proper output, but the converted model receives only 640x640 and I have some not found faces. E.g. 2560x1440 pixel frame and model could not find faces of approx. 60x35 size and couldn't find some slightly turned faces (the original model finds them ok). What can I do to convert the model to be saved with dynamic input size?

how to use triton_backend

Like this,can you tell me how to use triton_backend to deal with results

inference and comparing faces time in real time applications

Hi. I can do face recognition in real time using the Python insightface package and onnx pre-trained models.
(https://github.com/deepinsight/insightface/tree/master/python-package)
I really face a lot of questions and challenges if you please help me.

cuda 11.1
mxnet :1.8.1 from source installed
onnxruntime-gpu:1.7.0
numpy:1.17.0

I use the following code to identify faces for 1000 feature extracted faces:

import cv2
import sys
import time
import numpy as np
import insightface
from insightface.app import FaceAnalysis
from imutils import paths
import os
import pickle
from numpy as np
from numpy.linalg import norm


parser = argparse.ArgumentParser()
parser.add_argument('--ctx', default=0, type=int, help='GPU')
args = parser.parse_args()
app = FaceAnalysis(name='models')
app.prepare(ctx_id=args.ctx, det_size=(640, 640))

database = {}

def compare(feat1, feat2):

    distance = np.dot(feat2, feat1) / norm(feat2) * norm(feat1))

    return distance

verification_threshhold = 0.3
database = {}

data = pickle.loads(open('encodings.pickle', "rb").read())

encoding = np.empty(512, )
imagePaths = list(paths.list_images('dataset'))

for (i, imagePath) in enumerate(imagePaths):
       
       name = os.path.splitext(os.path.basename(imagePath))[0]
       image = cv2.imread(imagePath)

        t1 = time.time()
        encodings = app.get(frame)
        t2 = time.time()
        print('elapsed time for extract encodings: ',(t2-t1))

        ts = time.time()
        for num, e in enumerate(encodings):

            encoding = e.embedding

            identity = ''
            for (name, db_enc) in data.items():

                dist = compare(encoding, db_enc)

                if 0.35 < dist:
                    identity = name
        te = time.time()
        print('elapsed time for  compare face in 1000 data')

Here I use different images for identification and each image has a different number of faces.
For example, one image has one face and the other image has 6 faces and the other has 15 faces.
After testing different images, I came up with the following outputs:

picture 1 (1 face ) >>> elapsed time for extract encodings: 0.019 s , elapsed time for compare face in 1000 data: 0.009 s
picture 2 (6 faces ) >>> elapsed time for extract encodings: 0.057 s , elapsed time for compare face in 1000 data: 0.05 s
picture 2 (15 faces ) >>> elapsed time for extract encodings: 0.19 s , elapsed time for compare face in 1000 data: 0.22 s

This is not at all useful for my purpose, considering real-time detection using multiple cameras with at least 20,000 faces for comparison.
How can I reduce this time?
Am I doing the right thing?
Thank you in advance for your reply

Accuracy loss ?

Hi,

Is there any accuracy decrease after conversion from standard model ?

FileNotFoundError: [Errno 2] No such file or directory: '/models/onnx/retinaface_r50_v1/retinaface_r50_v1.onnx'``

Hi Team, any chance you know how this one happen
I really appreciate that
Thank you so much

return future.result()

File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 55, in serv
config.load()

File "/usr/local/lib/python3.8/dist-packages/uvicorn/config.py", line 308, in load

self.loaded_app = import_from_string(self.app)

File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 20, in import_from_string

module = importlib.import_module(module_str)

File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "", line 1014, in _gcd_import

File "", line 991, in _find_and_load

File "", line 975, in _find_and_load_unlocked

File "", line 671, in _load_unlocked

File "", line 783, in exec_module

File "", line 219, in _call_with_frames_removed

File "./app.py", line 34, in

processing = Processing(det_name=configs.models.det_name, rec_name=configs.models.rec_name,

File "./modules/processing.py", line 121, in init

self.model = FaceAnalysis(det_name=det_name, rec_name=rec_name, ga_name=ga_name, device=device,

File "./modules/face_model.py", line 69, in init

self.det_model = Detector(det_name=det_name, device=device, max_size=self.max_size,

File "./modules/face_model.py", line 35, in init

self.retina = get_model(det_name, backend_name=backend_name, force_fp16=force_fp16, im_size=max_size,

File "./modules/model_zoo/getter.py", line 175, in get_model

model_path = prepare_backend(model_name, backend_name, im_size=im_size, max_batch_size=max_batch_size, config=config, force_fp16=force_fp16,

File "./modules/model_zoo/getter.py", line 115, in prepare_backend

model = onnx.load(onnx_path)

File "/usr/local/lib/python3.8/dist-packages/onnx/init.py", line 114, in load_model

s = _load_bytes(f)

File "/usr/local/lib/python3.8/dist-packages/onnx/init.py", line 30, in _load_bytes

with open(cast(Text, f), 'rb') as readable:

FileNotFoundError: [Errno 2] No such file or directory: '/models/onnx/retinaface_r50_v1/retinaface_r50_v1.onnx'

Preparing models...

[22:14:46] INFO - Preparing 'arcface_r100_v1' model...

[22:14:46] INFO - Reshaping ONNX inputs to: (1, 3, 112, 112)

[22:14:46] INFO - Building TRT engine for arcface_r100_v1...

mxnet version: 1.7.0

onnx version: 1.7.0

Traceback (most recent call last):

File "prepare_models.py", line 45, in

prepare_models()

File "prepare_models.py", line 40, in prepare_models

prepare_backend(model_name=model, backend_name=backend_name, im_size=max_size, force_fp16=force_fp16,

File "/app/modules/model_zoo/getter.py", line 126, in prepare_backend

convert_onnx(temp_onnx_model,

TypeError: 'NoneType' object is not callable

Starting InsightFace-REST using 1 workers.

[22:14:47] INFO - Reshaping ONNX inputs to: (1, 3, 640, 640)

mxnet version: 1.7.0

onnx version: 1.7.0

Traceback (most recent call last):

File "/usr/local/bin/uvicorn", line 8, in

sys.exit(main())

File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 829, in call

return self.main(*args, **kwargs)

File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 782, in main

rv = self.invoke(ctx)

File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1066, in invoke

return ctx.invoke(self.callback, **ctx.params)

File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 610, in invoke

return callback(*args, **kwargs)

File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 362, in main

run(**kwargs)

File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 386, in run

server.run()

File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 48, in run

loop.run_until_complete(self.serve(sockets=sockets))

File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete

return future.result()

File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 55, in serve

config.load()

File "/usr/local/lib/python3.8/dist-packages/uvicorn/config.py", line 308, in load

self.loaded_app = import_from_string(self.app)

File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 20, in import_from_string

module = importlib.import_module(module_str)

File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "", line 1014, in _gcd_import

File "", line 991, in _find_and_load

File "", line 975, in _find_and_load_unlocked

File "", line 671, in _load_unlocked

File "", line 783, in exec_module

File "", line 219, in _call_with_frames_removed

File "./app.py", line 34, in

processing = Processing(det_name=configs.models.det_name, rec_name=configs.models.rec_name,

File "./modules/processing.py", line 121, in init

self.model = FaceAnalysis(det_name=det_name, rec_name=rec_name, ga_name=ga_name, device=device,

File "./modules/face_model.py", line 69, in init

self.det_model = Detector(det_name=det_name, device=device, max_size=self.max_size,

File "./modules/face_model.py", line 35, in init

self.retina = get_model(det_name, backend_name=backend_name, force_fp16=force_fp16, im_size=max_size,

File "./modules/model_zoo/getter.py", line 175, in get_model

model_path = prepare_backend(model_name, backend_name, im_size=im_size, max_batch_size=max_batch_size, config=config, force_fp16=force_fp16,

File "./modules/model_zoo/getter.py", line 115, in prepare_backend

model = onnx.load(onnx_path)

File "/usr/local/lib/python3.8/dist-packages/onnx/init.py", line 114, in load_model

s = _load_bytes(f)

File "/usr/local/lib/python3.8/dist-packages/onnx/init.py", line 30, in _load_bytes

with open(cast(Text, f), 'rb') as readable:

FileNotFoundError: [Errno 2] No such file or directory: '/models/onnx/retinaface_r50_v1/retinaface_r50_v1.onnx'``

Failed to create arcface tensorrt model with tensorrt 20.10

Hi, I was trying to convert Arcface model to TensorRT plan using build_insight_trt.py and TensorRT 20.10, but it failed with the following error-

[TensorRT] ERROR: (Unnamed Layer* 480) [Shuffle]: at most one dimension may be inferred
ERROR: Failed to parse the ONNX file: /models/onnx/arcface_r100_v1/arcface_r100_v1.onnx.tmp
In node -1 (scaleHelper): UNSUPPORTED_NODE: Assertion failed: dims.nbDims == 4 || dims.nbDims == 5

Require this because Triton supports custom Python backends only with Triton 20.10 and not 20.09, and the model converted using TensorRT 20.09 is unsupported with Triton 20.10 (Triton 20.10 supports TensorRT 20.10)

Can this be fixed?

Also, Thanks for the amazing work!

Method not allowed

When I try to call the API using postman, I am getting this error.

<title>405 Method Not Allowed</title>

Method Not Allowed

The method is not allowed for the requested URL.

Please let me know what can be done. I am using converted base64 images and adding in JSON requests like "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQA.........."

Similarity formula needs explanation

Please explain the following formula:

sim = (1. + np.dot(emb1, emb2)) / 2

why using this instead of cosine similarity?

arcface: TensorRT model having lower accuracy than original mxnet model?

Plotting positive & negative pairs simiilarity-scores histogram, I am seeing some drop in accuracy of TensorRT model. Is anyone also seeing it

In order to fix missing arguments, trt_backend and onnxrt_backend should also add **kwargs like triton_backend?

different croped face from draw_detections and extract Api for same image

Hi..
I was doing testing on the diiferent api's of the repo & realised one thing that there is huge differnece between bouding box that has been drwan around face using /draw_detections and face data written from /extract api while setting return_face_data = True for the same image.I am getting very low performace on images so I started dig down in the code and i realised that while getting embedding from model you are not using bounding box area instead you are making affine transformation around landmarks so any particular reason for this why not bounding box ?

for reference i have added images you can see there is extra pixels in last image compare to first image

Confuse about arcface_r100_v1.plan

@SthPhoenix I have converted the mxnet model to tensorrt model . I get the arcface_r100_v1.plan as a result, is it tensorrt model or something else becuase tensorrt model have extension .trt as I have seen on the repository

https://github.com/jkjung-avt/tensorrt_demos

Could you explain it for me.Thanks

onnx

Hi SthPhoenix,

How to run the model to onnx?
https://github.com/deepinsight/insightface/tree/master/recognition/partial_fc

thanks

CenterFace conversion script failure

when running build_centerface_trt.py:

[TensorRT] ERROR: _107: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [1,24,136,240] and [1,24,135,240]).
[TensorRT] ERROR: _107: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [1,24,136,240] and [1,24,135,240]).
[TensorRT] ERROR: _107: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [1,24,136,240] and [1,24,135,240]).
ERROR: Failed to parse the ONNX file: /models/onnx/centerface/centerface.onnx.temp
In node 109 (importConvTranspose): UNSUPPORTED_NODE: Assertion failed: dims.nbDims >= 3 && dims.nbDims <= 5 && "TensorRT only supports 1D, 2D or 3D deconvolutions!"

Batch inference detection model

I’m trying to convert retinaface_mnet025_v1 onnx model to tensorrt with batchsize 64.
During inference,it takes 390ms for batch inference (batch_size = 64)
and 7 ms for batch size = 1.

Please let me know if I’m missing out on something.
Attaching google drive link for tensorrt model generation.
here

scrfd_ * shape issue

Hello,

for models scrfd_* if i try out "test_images/Stallone.jpg"

docker run -p 18080:18080 -it --gpus '"device=0"' -e LOG_LEVEL=INFO -e PYTHONUNBUFFERED=0 -e
NUM_WORKERS=1 -e INFERENCE_BACKEND=trt -e FORCE_FP16=True -e DET_NAME=scrfd_10g_gnkps -e DET_THRESH=0.6 -e REC_NAME=glint36
0k_r100FC_1.0 -e REC_IGNORE=False -e REC_BATCH_SIZE=1 -e GA_NAME=genderage_v1 -e GA_IGNORE=False -e KEEP_ALL=True -e MAX_SIZE
=1024,780 -e DEF_RETURN_FACE_DATA=True -e DEF_EXTRACT_EMBEDDING=True -e DEF_EXTRACT_GA=True -e DEF_API_VER='1' --mount type=b
ind,source=/home/work/services/models,target=/models --health-cmd='curl -f http://localhost:18080/info || exit 1' --health-i
nterval=1m --health-timeout=10s --health-retries=3 insightface-rest
Preparing models...
mxnet version: 1.8.0
onnx version: 1.7.0
[08:23:40] INFO - Preparing 'glint360k_r100FC_1.0' model...
[08:23:40] INFO - Preparing 'scrfd_10g_gnkps' model...
[08:23:40] INFO - Reshaping ONNX inputs to: (1, 3, 780, 1024)
[08:23:40] INFO - Building TRT engine for scrfd_10g_gnkps...
[08:23:52] INFO - Building TensorRT engine with FP16 support.
[TensorRT] WARNING: /workspace/TensorRT/t/oss-cicd/oss/parsers/onnx/onnx2trt_utils.cpp:227: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: No implementation obeys reformatting-free rules, at least 10 reformatting nodes are needed, now picking the fastest path instead.
[08:24:46] INFO - Building TRT engine complete!
[08:24:46] INFO - Preparing 'genderage_v1' model...
Starting InsightFace-REST using 1 workers.
mxnet version: 1.8.0
onnx version: 1.7.0
[08:24:49] INFO - Warming up face detector TensorRT engine...
[08:25:02] INFO - Engine warmup complete! Expecting input shape: (1, 3, 780, 1024)
[08:25:02] INFO - Warming up ArcFace TensorRT engine...
[08:25:03] INFO - Engine warmup complete! Expecting input shape: (1, 3, 112, 112). Max batch size: 1
[08:25:03] INFO - Warming up GenderAge TensorRT engine...
[08:25:03] INFO - Engine warmup complete! Expecting input shape: (1, 3, 112, 112). Max batch size: 1
INFO: Started server process [233]
[08:25:03] INFO - Started server process [233]
INFO: Waiting for application startup.
[08:25:03] INFO - Waiting for application startup.
INFO: Application startup complete.
[08:25:03] INFO - Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:18080 (Press CTRL+C to quit)
[08:25:03] INFO - Uvicorn running on http://0.0.0.0:18080 (Press CTRL+C to quit)
INFO: 127.0.0.1:35790 - "GET /info HTTP/1.1" 200 OK
INFO: 192.168.49.168:65047 - "GET /docs HTTP/1.1" 200 OK
INFO: 192.168.49.168:65047 - "GET /openapi.json HTTP/1.1" 200 OK
inference cost: 0.0219266414642334
Traceback (most recent call last):
File "./modules/processing.py", line 236, in embed
faces = await self.model.get(image, max_size=max_size, threshold=threshold,
File "./modules/face_model.py", line 189, in get
boxes, probs, landmarks, mask_probs = self.det_model.detect(img.transformed_image, threshold=threshold)
File "./modules/face_model.py", line 40, in detect
bboxes, landmarks = self.retina.detect(data, threshold=threshold)
File "./modules/model_zoo/detectors/scrfd.py", line 173, in detect
scores_list, bboxes_list, kpss_list = self.forward(img, threshold)
File "./modules/model_zoo/detectors/scrfd.py", line 159, in forward
bboxes = distance2bbox(anchor_centers, bbox_preds)
File "./modules/model_zoo/detectors/scrfd.py", line 39, in distance2bbox
x1 = points[:, 0] - distance[:, 0]
ValueError: operands could not be broadcast together with shapes (24832,) (25088,)

what could be a solution?
NVIDIA-SMI 465.19.01
Driver Version: 465.19.01
CUDA Version: 11.3
NVIDIA Tesla T4

if change to retinaface_r50_v1 everything works fine.

i try to change base container to nvcr.io/nvidia/tensorrt:21.06-py3 but nothing changes

Preparing models...
mxnet version: 1.8.0
onnx version: 1.7.0
[07:53:24] INFO - Preparing 'glint360k_r100FC_1.0' model...
[07:53:24] INFO - Preparing 'scrfd_10g_gnkps' model...
[07:53:24] INFO - Reshaping ONNX inputs to: (1, 3, 780, 1024)
[07:53:24] INFO - Building TRT engine for scrfd_10g_gnkps...
[07:53:36] INFO - Building TensorRT engine with FP16 support.
[TensorRT] WARNING: /home/jenkins/agent/workspace/OSS/OSS_L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:271: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_130 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_177 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_141 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_224 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_188 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_152 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_235 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_199 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation of layer InstanceNormalization_246 obeys the requested constraints in strict mode. No conforming implementation was found i.e. requested layer computation precision and output precision types are ignored, using the fastest implementation.
[TensorRT] WARNING: No implementation obeys reformatting-free rules, at least 10 reformatting nodes are needed, now picking the fastest path instead.
[07:54:32] INFO - Building TRT engine complete!
[07:54:32] INFO - Preparing 'genderage_v1' model...
Starting InsightFace-REST using 1 workers.
mxnet version: 1.8.0
onnx version: 1.7.0
[07:54:36] INFO - Warming up face detector TensorRT engine...
[07:54:48] INFO - Engine warmup complete! Expecting input shape: (1, 3, 780, 1024)
[07:54:48] INFO - Warming up ArcFace TensorRT engine...
[07:54:49] INFO - Engine warmup complete! Expecting input shape: (1, 3, 112, 112). Max batch size: 1
[07:54:49] INFO - Warming up GenderAge TensorRT engine...
[07:54:49] INFO - Engine warmup complete! Expecting input shape: (1, 3, 112, 112). Max batch size: 1
INFO:     Started server process [233]
[07:54:49] INFO - Started server process [233]
INFO:     Waiting for application startup.
[07:54:49] INFO - Waiting for application startup.
INFO:     Application startup complete.
[07:54:49] INFO - Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:18080 (Press CTRL+C to quit)
[07:54:49] INFO - Uvicorn running on http://0.0.0.0:18080 (Press CTRL+C to quit)
INFO:     192.168.49.168:64156 - "GET /docs HTTP/1.1" 200 OK
INFO:     192.168.49.168:64156 - "GET /openapi.json HTTP/1.1" 200 OK
inference cost: 0.02373218536376953
Traceback (most recent call last):
  File "./modules/processing.py", line 236, in embed
    faces = await self.model.get(image, max_size=max_size, threshold=threshold,
  File "./modules/face_model.py", line 189, in get
    boxes, probs, landmarks, mask_probs = self.det_model.detect(img.transformed_image, threshold=threshold)
  File "./modules/face_model.py", line 40, in detect
    bboxes, landmarks = self.retina.detect(data, threshold=threshold)
  File "./modules/model_zoo/detectors/scrfd.py", line 173, in detect
    scores_list, bboxes_list, kpss_list = self.forward(img, threshold)
  File "./modules/model_zoo/detectors/scrfd.py", line 159, in forward
    bboxes = distance2bbox(anchor_centers, bbox_preds)
  File "./modules/model_zoo/detectors/scrfd.py", line 39, in distance2bbox
    x1 = points[:, 0] - distance[:, 0]
ValueError: operands could not be broadcast together with shapes (24832,) (25088,)

layer InstanceNormalization was fix in new TRT but there is no prebuild nvidia container yet

batch inference arcface model with tensorrt

I'm trying to convert my onnx model from static input(1,3,112,112) to dynamic input(?,3,112,112) so that I can run the inference with batchsize. I see in your project that the function add_dyn_batch (in src/converters/modules/converters/insight2onnx.py) seems to do this before converting the model to trt. And I tried using that function to modify the dynamic input for the onnx model, the onnx model's static to dynamic input change worked, but when I used that modified model to convert to trt there was an error happening:

Error when parsing ONNX file

[TensorRT] ERROR: (Unnamed Layer * 480) [Shuffle]: at most one dimension may be inferred

Error when building an engine

[TensorRT] ERROR: Network must have at least one output

P/s: I have done the conversion from onnx-> trt with static input (1,3,112,112)
Do you have a solution for this problem?
Thank you very much

这是个bug吗

INFO: Uvicorn running on http://0.0.0.0:18080 (Press CTRL+C to quit)
[15:44:52] INFO - Uvicorn running on http://0.0.0.0:18080 (Press CTRL+C to quit)

Traceback (most recent call last):
File "/data/service/InsightFace-REST/src/api_trt/./modules/processing.py", line 208, in embed_crops
_face_dict = serialize_face(face=next(faces), return_face_data=False,
File "/data/service/InsightFace-REST/src/api_trt/./modules/face_model.py", line 133, in process_faces
embeddings = self.rec_model.get_embedding(crops)
File "/data/service/InsightFace-REST/src/api_trt/./modules/model_zoo/exec_backends/trt_backend.py", line 74, in get_embedding
embeddings = self.rec_model.run(face_img, deflatten=True)[0]
File "/data/service/InsightFace-REST/src/api_trt/./modules/model_zoo/exec_backends/trt_loader.py", line 101, in run
self.inputs[0].host[:allocate_place] = input.flatten(order='C').astype(np.float32)
ValueError: could not broadcast input array from shape (177540,) into shape (37632,)

INFO: 172.21.110.251:52523 - "POST /extract HTTP/1.1" 200 OK

请求参数如下
{
"images": {
"data": [
"string"
],
"urls": [
"test_images/Stallone.jpg"
]
},
"max_size": [
640,
640
],
"threshold": 0.6,
"embed_only": true,
"return_face_data": false,
"return_landmarks": false,
"extract_embedding": true,
"extract_ga": false,
"limit_faces": 1,
"verbose_timings": true,
"api_ver": "1"
}

Batch inference with recognition model

Hi. In an image in which face detector detects multiple faces, I want to feed all of them to the recognition model at once instead of one by one using a for loop. I'm using trt backend. Can you give me a guide for doing that?

Clarification on running the service on just cpu

Hi,

Thank you for this amazing API.

I am trying to run the service on a VM that does not have gpu. Based on the readme, I used the 0.5 version and removed --gpu settings in docker run as well as changed the device to 'cpu' in fapp.py. While the container starts running without issues, I cannot see the documentation on local host:18081

Am I missing any other changes? I am a bit new to docker. I do not know if this port is mapped correctly or not.

Once again, thank you for your help.
Gokul

Cannot change REC_BATCH_SIZE to other number instead of 1？

ERROR: Failed to parse the ONNX
In node -1 (scaleHelper): UNSUPPORTED_NODE: Assertion failed: dims.nbDims == 4 || dims.nbDims == 5
tensorrt version: 7.2.1.6
onnx version: 1.7.0

how to start

how to use insight2onnx.py

python src/converters/modules/converters/insight2onnx.py

Traceback (most recent call last):
File "src/converters/modules/converters/insight2onnx.py", line 13, in
from .mx2onnx_conv import onnx as onnx_mxnet
ModuleNotFoundError: No module named 'main.mx2onnx_conv'; 'main' is not a package

Error when building converter docker image

When trying to run ./deploy_converter.sh:

"docker stop" requires at least 1 argument.
See 'docker stop --help'.

Usage:  docker stop [OPTIONS] CONTAINER [CONTAINER...]

Stop one or more running containers
"docker rm" requires at least 1 argument.
See 'docker rm --help'.

Usage:  docker rm [OPTIONS] CONTAINER [CONTAINER...]

Remove one or more containers
[+] Building 718.9s (5/12)                                                                                                                                                                  
 => [internal] load .dockerignore                                                                                                                                                      0.0s
 => => transferring context: 2B                                                                                                                                                        0.0s
 => [internal] load build definition from Dockerfile.converter                                                                                                                         0.0s
 => => transferring dockerfile: 550B                                                                                                                                                   0.0s
 => [internal] load metadata for nvcr.io/nvidia/tensorrt:20.09-py3                                                                                                                     5.1s
 => ERROR [1/8] FROM nvcr.io/nvidia/tensorrt:20.09-py3@sha256:af35aa6b82c40e659d34c33596996221ec05f8cb4d7dbcd5a8a501d351db2dbd                                                       713.7s
 => => resolve nvcr.io/nvidia/tensorrt:20.09-py3@sha256:af35aa6b82c40e659d34c33596996221ec05f8cb4d7dbcd5a8a501d351db2dbd                                                               0.0s
 => => sha256:af35aa6b82c40e659d34c33596996221ec05f8cb4d7dbcd5a8a501d351db2dbd 9.73kB / 9.73kB                                                                                         0.0s
 => => sha256:b1386993f5711a7b77bfdd390f72be8155541f007449ef2c73265d6ec2f68960 29.85kB / 29.85kB                                                                                       0.0s
 => => sha256:13cf13e5ce72f7ff8419042d6d9ea22fa94c18b94c4985269339bb7eaa229e9a 18.82kB / 18.82kB                                                                                       1.3s
 => => sha256:ffcd5dc3448dfb808c9117252c4f1fbbeb20128707d7b9c6b53c29d6f2b988a1 50.59MB / 50.59MB                                                                                     171.4s
 => => sha256:c6b364205fad9034ce6cbf439da0d4a7c0cc290b4c8d3ad1eabfc81b04b7e8b3 76.81MB / 76.81MB                                                                                     359.1s
 => => sha256:7202bec79e4143bda783391ae8cbaf210306c344350f892e9ffd77ba218cf063 2.73kB / 2.73kB                                                                                         3.0s
 => => sha256:fdfbe893941b3876d9e8c3e411178b3bf9f4a316363a98b9181a2c46c6f5e2f2 3.63kB / 3.63kB                                                                                         4.1s
 => => sha256:f73bfa0e0e1718cab44daf35ff2045a57df7d993f83812c95916bb0c42d081e0 137B / 137B                                                                                             5.2s
 => => sha256:36aade146566f729b04737087ae3302fb8bfca68f02b74f4ba542c53da41bbe6 1.04kB / 1.04kB                                                                                         6.5s
 => => sha256:0cf8254e1bfeb99899238126562204641d0152a417c5f75c8457a29fda84cc10 397B / 397B                                                                                             7.6s
 => => sha256:40ff6c34e5e5179007060a7d9b0916d80e827ad0eac05933d13f4d9b94546b1b 429B / 429B                                                                                             8.7s
 => => sha256:0adeec2cfe741e924ea7c35ca95d6dc0fb08861268e50415e7153851715cabd9 1.44kB / 1.44kB                                                                                         9.9s
 => => sha256:895d871af5fda54cf858b531272369c3d42c63ae37da075ae7d32f71ddfd9c16 184.34MB / 1.90GB                                                                                     713.3s
 => => sha256:71c97f6ac83cefd4d6c3020f909de20166e25ad0fcab9c19b519c8aa0190e2d2 11.56kB / 11.56kB                                                                                     173.9s
 => => sha256:281aa21cb81222304da410b309545b393c8daf5281d6440de38a15bb7cd84ab9 186B / 186B                                                                                           175.1s
 => => sha256:9c7e46bb4080abaae52fac2d3dba4ffffe2383a60911071c74430220e22d1caf 172.73kB / 172.73kB                                                                                   176.8s
 => => sha256:150dfb1677cd0f25666657bdbc203c7f638e50bcbfd986815ac0b6d73d2120a9 736B / 736B                                                                                           177.9s
 => => sha256:3ce488e63cd39ebbb9c42f581c4fe24feb5b57f9220694e8199714ead733397f 37.24MB / 37.24MB                                                                                     346.5s
 => => sha256:0f6e3807a6dc22c33448ed4127391b494656eca8ea3e6ddbc4b8c500fc44739d 7.82MB / 7.82MB                                                                                       387.9s
 => => extracting sha256:c6b364205fad9034ce6cbf439da0d4a7c0cc290b4c8d3ad1eabfc81b04b7e8b3                                                                                              1.0s
 => => sha256:3585c705a7d04fde836f985ac28893d8207325669f359f961ed9a8aa15c94b89 568.03kB / 568.03kB                                                                                   363.6s
 => => extracting sha256:ffcd5dc3448dfb808c9117252c4f1fbbeb20128707d7b9c6b53c29d6f2b988a1                                                                                              0.6s
 => => extracting sha256:13cf13e5ce72f7ff8419042d6d9ea22fa94c18b94c4985269339bb7eaa229e9a                                                                                              0.0s
 => => extracting sha256:7202bec79e4143bda783391ae8cbaf210306c344350f892e9ffd77ba218cf063                                                                                              0.0s
 => => extracting sha256:fdfbe893941b3876d9e8c3e411178b3bf9f4a316363a98b9181a2c46c6f5e2f2                                                                                              0.0s
 => => extracting sha256:f73bfa0e0e1718cab44daf35ff2045a57df7d993f83812c95916bb0c42d081e0                                                                                              0.0s
 => => extracting sha256:36aade146566f729b04737087ae3302fb8bfca68f02b74f4ba542c53da41bbe6                                                                                              0.0s
 => => extracting sha256:0cf8254e1bfeb99899238126562204641d0152a417c5f75c8457a29fda84cc10                                                                                              0.0s
 => => extracting sha256:40ff6c34e5e5179007060a7d9b0916d80e827ad0eac05933d13f4d9b94546b1b                                                                                              0.0s
 => => extracting sha256:0adeec2cfe741e924ea7c35ca95d6dc0fb08861268e50415e7153851715cabd9                                                                                              0.0s
 => => sha256:de81ad6998228a35c0dea2dffddb0845562ba941a1bde89e6a3a05f2606b52be 12.84kB / 12.84kB                                                                                     365.3s
 => => sha256:bb1a224031d930789559694e1dab05fe12dda7f18b33b2e8eeb915d714af3e15 11.96MB / 11.96MB                                                                                     374.6s
 => => sha256:de7308abc9b3a5397bdf18f3e2aefcc6c8609aef730327b0f825a35070a46112 432B / 432B                                                                                           375.9s
 => => sha256:07620b1781c27a8f08b59cc4757797049648490f14f5d92984e97eea8c359d1b 512B / 512B                                                                                           377.3s
 => => sha256:dbc3331c85c9801958b1136996a594b5eed49b40debcebd80b26d2b70c727fea 76.92MB / 217.85MB                                                                                    713.3s
 => => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B                                                                                             389.2s
 => => sha256:0e2041941e1021553683d86a9b2a1f9db7d463cb5aac534c2f2be2d0169348d6 59.13MB / 59.13MB                                                                                     657.2s
 => => sha256:04c1de6b1c97e8c720d3012dde43ea85700c296598afe0c3f9fbf18b5beb1c9e 0B / 13.63MB                                                                                          713.3s
 => [internal] load build context                                                                                                                                                      0.9s
 => => transferring context: 262.69MB                                                                                                                                                  0.9s
------
 > [1/8] FROM nvcr.io/nvidia/tensorrt:20.09-py3@sha256:af35aa6b82c40e659d34c33596996221ec05f8cb4d7dbcd5a8a501d351db2dbd:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: failed to copy: httpReaderSeeker: failed open: failed to fetch anonymous token: unexpected status: 401 Unauthorized
Unable to find image 'insightface-converter:v0.1' locally
docker: Error response from daemon: pull access denied for insightface-converter, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

cannot start container from ./deploy_trt.sh

ERROR: CUDA initialization failure with error 222. Please check your CUDA installation: http://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
mxnet version: 1.7.0
onnx version: 1.7.0
Traceback (most recent call last):
File "prepare_models.py", line 45, in
prepare_models()
File "prepare_models.py", line 40, in prepare_models
prepare_backend(model_name=model, backend_name=backend_name, im_size=max_size, force_fp16=force_fp16,
File "/app/modules/model_zoo/getter.py", line 127, in prepare_backend
convert_onnx(temp_onnx_model,
File "/app/modules/converters/onnx_to_trt.py", line 79, in convert_onnx
engine = _build_engine_onnx(input_onnx=onnx_obj,
File "/app/modules/converters/onnx_to_trt.py", line 25, in _build_engine_onnx
with trt.Builder(TRT_LOGGER) as builder,
TypeError: pybind11::init(): factory function returned nullptr
Starting InsightFace-REST using 1 workers.
[09:43:07] INFO - Reshaping ONNX inputs to: (1, 3, 640, 640)
mxnet version: 1.7.0
onnx version: 1.7.0

how to change diff. model in server?

first this repo. is awesome thx you~
but I don't know how to change
so that I can use different models to test

When i use plan in Triton Inference Server get a error

Can you help me how to transfet model to Triton Inference Server
Can you tell me when have Triton Inference Server examples

retinaface onnx inference

你好，不太清楚你转换retinaface成onnx的流程以及推论代码的位置，请问方便说说吗?

Consfuion About the Inference Script

@SthPhoenix I have studied the inference script , So I get the following understanding
-> You are loading an input image
-> Preparing and loading the Tensorrt engine
-> Preparing and loading the orignal mxnet model
-> You are getting embedding the from both model
-> Then comparing the result of both model embedding.

Cqn you give me briefly explanation.It will be helpful because I want to show results on opencv windows .Thanks

runs on windows 10

Hello. I want to run deploy_cpu.sh on Windows 10 Docker. But I have no idea. Is it possible?

Giving multiple detected faces at once to recognition model

hi. I am able to detect faces in an image with scrfd. If there is more than one face in the image, I have to give the faces one by one to the Recognition model, respectively.
How can I do this in such a way that I get all the faces in one go (at once ) and get the features in less time?

bboxes, kpss = self.det_model.detect(img,
                                             max_num=max_num,
                                             metric='default')
        if bboxes.shape[0] == 0:
            return []
        ret = []
        for i in range(bboxes.shape[0]):
            
            bbox = bboxes[i, 0:4]
            det_score = bboxes[i, 4]
            kps = None
            if kpss is not None:
                kps = kpss[i]
            face = Face(bbox=bbox, kps=kps, det_score=det_score)

            for taskname, model in self.models.items():

                if taskname=='detection':
                    continue
                model.get(img, face)
            ret.append(face)
            
        
        return ret

get feature :

def get_feat(self, imgs):
        if not isinstance(imgs, list):
            imgs = [imgs]
 
        input_size = self.input_size
        
        blob = cv2.dnn.blobFromImages(imgs, 1.0 / self.input_std, input_size,
                                      (self.input_mean, self.input_mean, self.input_mean), swapRB=True)
   
        net_out = self.session.run(self.output_names, {self.input_name: blob})[0]
   
        return net_out

Unable to process images which are received from socketio image buffer

hi @SthPhoenix :

if i run the code on the images which are there on my machine then it is working fine. However when I am trying to process the image which i receive from the socket connection , I am getting this error :

Will you be able to give me some hint in this case ?

[TensorRT] ERROR: ../rtSafe/cuda/reformat.cu (925) - Cuda Error in NCHWToNCHHW2: 400 (invalid resource handle)
[TensorRT] ERROR: FAILED_EXECUTION: std::exception
[TensorRT] ERROR: ../rtSafe/cuda/reformat.cu (925) - Cuda Error in NCHWToNCHHW2: 400 (invalid resource handle)
[TensorRT] ERROR: FAILED_EXECUTION: std::exception

Note :
Before sending it to inference I am following below steps :

#Model 
model = FaceAnalysis(max_size=[640, 640], backend_name='trt', det_name='retinaface_mnet025_v1', 
			rec_name='arcface_r100_v1',max_rec_batch_size=64)

#here i recv the image
recvdImg  =  data.get("frame")

#to numpy array
nparr = np.frombuffer(recvdImg,dtype=np.uint8,offset=1)

#decoding the image
img_np = cv2.imdecode(nparr,-1)

# doing inferece :
faces = model.get(img_np)

Multiple Cuda Context and multi threading

hi @SthPhoenix :

Can you give some hint in creating of the below use case:

I want to now create 2 different cuda context ! I am having sufficient GPU memory which can hold both the context.

Now in this case ,

I will receive frame in queue and every alternate frame will be processed by single cuda context.

Any headway ??

RetinaFaceAntiCov TRT Inference

Hello. I'm trying to use this pretrained insightface model with tensorrt. Successfully converted into trt format but having trouble when making predictions. I keep the original retina inference code. Maybe this model has different output shape then it causes an error:

File "/FaceRecognize/src/converters/modules/model_zoo/detectors/retinaface.py", line 322, in postprocess
    landmark_deltas = landmark_deltas.transpose((0, 2, 3, 1)).reshape((-1, 5, landmark_pred_len // 5))
ValueError: cannot reshape array of size 16320 into shape (5,0)

Could you please give me a hand to solve this?

Use conversion for Tensorflow

Hi,
Do you think it could be possible to use your onnx fixes to be able to convert it to a tensorflow model, not RT.

Thank you,

convert_onnx error

[08:22:52] INFO - Preparing 'arcface_r100_v1' model...
[08:22:52] INFO - Building TRT engine for arcface_r100_v1...
mxnet version: 1.7.0
onnx version: 1.7.0
Traceback (most recent call last):
File "prepare_models.py", line 45, in
prepare_models()
File "prepare_models.py", line 40, in prepare_models
prepare_backend(model_name=model, backend_name=backend_name, im_size=max_size, force_fp16=force_fp16,
File "/app/modules/model_zoo/getter.py", line 126, in prepare_backend
convert_onnx(temp_onnx_model,
TypeError: 'NoneType' object is not callable
Starting InsightFace-REST using 1 workers.
[08:22:54] INFO - Reshaping ONNX inputs to: (1, 3, 640, 640)
mxnet version: 1.7.0
onnx version: 1.7.0
Traceback (most recent call last):
File "/usr/local/bin/uvicorn", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 362, in main
run(**kwargs)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/main.py", line 386, in run
server.run()
File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 48, in run
loop.run_until_complete(self.serve(sockets=sockets))
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/usr/local/lib/python3.8/dist-packages/uvicorn/server.py", line 55, in serve
config.load()
File "/usr/local/lib/python3.8/dist-packages/uvicorn/config.py", line 308, in load
self.loaded_app = import_from_string(self.app)
File "/usr/local/lib/python3.8/dist-packages/uvicorn/importer.py", line 20, in import_from_string
module = importlib.import_module(module_str)
File "/usr/lib/python3.8/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in _find_and_load_unlocked
File "", line 671, in _load_unlocked
File "", line 783, in exec_module
File "", line 219, in _call_with_frames_removed
File "./app.py", line 34, in
processing = Processing(det_name=configs.models.det_name, rec_name=configs.models.rec_name,
File "./modules/processing.py", line 121, in init
self.model = FaceAnalysis(det_name=det_name, rec_name=rec_name, ga_name=ga_name, device=device,
File "./modules/face_model.py", line 69, in init
self.det_model = Detector(det_name=det_name, device=device, max_size=self.max_size,
File "./modules/face_model.py", line 35, in init
self.retina = get_model(det_name, backend_name=backend_name, force_fp16=force_fp16, im_size=max_size,
File "./modules/model_zoo/getter.py", line 175, in get_model
model_path = prepare_backend(model_name, backend_name, im_size=im_size, max_batch_size=max_batch_size, config=config, force_fp16=force_fp16,
File "./modules/model_zoo/getter.py", line 115, in prepare_backend
model = onnx.load(onnx_path)
File "/usr/local/lib/python3.8/dist-packages/onnx/init.py", line 114, in load_model
s = _load_bytes(f)
File "/usr/local/lib/python3.8/dist-packages/onnx/init.py", line 30, in _load_bytes
with open(cast(Text, f), 'rb') as readable:
FileNotFoundError: [Errno 2] No such file or directory: '/models/onnx/retinaface_r50_v1/retinaface_r50_v1.onnx'

sthphoenix / insightface-rest Goto Github PK

insightface-rest's People

Contributors

Stargazers

Watchers

Forkers

insightface-rest's Issues

Request:

Error

Method Not Allowed

Error when parsing ONNX file

Error when building an engine

Recommend Projects

Recommend Topics

Recommend Org