spacewalk01 / tensorrt-openpose Goto Github PK

TensorRT C++ Implementation of openpose

License: MIT License

Python 3.30% Cuda 2.76% C++ 93.58% C 0.35%

pose-estimation object-detection tensorrt deep-learning openpose human-pose-estimation real-time-object-detection cpp

tensorrt-openpose's Introduction

TensorRT Implementation of OpenPose

This repo provides the C++ implementation of OpenPose for doing real-time pose estimation on Windows platform. It is based on pose detection program developed by NVIDIA and performs pose estimation on TensorRT framework with a high throughtput. The pose estimation program runs at up to 500 FPS on RTX-3070 GPU using 224x224 ResNet18.

Requirements

The following environment was set for the experiment but if you have a different Graphic Card, you need to download and install TensorRT / CUDA that matches your GPU version.

Windows 10
Visual Studio 2017
RTX 3070 GPU
TensorRT 7.2.1.6
CUDA 11.1, Cudnn 8
Python 3.7
Torch 1.8.1
OpenCV 4.5.1 with CUDA

Installation

Install PyTorch

pip3 install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio===0.8.1 -f https://download.pytorch.org/whl/torch_stable.html

Download and install CUDA and CUDNN by following this installation guide.
Install [OpenCV 4.5.1 or 4.5.0]
Download the TensorRT zip file that matches your Windows version.
Install TensorRT by copying the DLL files from <tensorrt_path>/lib to your CUDA installation directory. For more information, refer to TensorRT installation guideline.

move <TensorRT_Installpath>/lib/*.dll "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1/bin"

Install trt_pose for parsing the trained pytorch model to an onnx graph model. For more information, refer to trt_pose installation guide.

git clone https://github.com/NVIDIA-AI-IOT/trt_pose
cd trt_pose
python setup.py install

Clone this github repository

cd ..
git clone https://github.com/spacewalk01/TensorRT-Openpose
cd Human_pose_detection

Download a pretrained resnet model, resnet18_baseline_att_224x224_A and put it in the current project folder.
Create an onnx model, optimize the model using TensorRT and build a runtime inference engine.

python convert2onnx.py -i resnet18_baseline_att_224x224_A_epoch_249.pth -o trt_pose.onnx
<tensorrt_path>/bin/trtexec.exe --onnx=trt_pose.onnx --explicitBatch --saveEngine=trt_pose_fp16.engine --fp16

Open the solution with Visual Studio. Select x64 and Release for the configuration and start building the project.

Usage

#include <opencv2/core.hpp>
#include "opencv2/imgcodecs.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include "Openpose.h"
#include "TensorrtPoseNet.h"

int main() {

    std::string filepath = "your image path";
   
    // Initialize pose network
    TensorrtPoseNet posenet("trt_pose_fp16.engine", 0.9, 0.1);
    
    // Initialize openpose parser
    Openpose openpose(posenet.outputDims[0]);

    cv::Mat image = cv::imread(filepath);

    // First, openpose uses resnet to predict a heatmap for each keypoint on the image
    net.infer(image);  

    // Then, it groups the keypoints to estimate a skeleton pose for each person
    openpose.detect(posenet.cpuCmapBuffer, posenet.cpuPafBuffer, image); 
    
    cv::imshow("Result", image);
    cv::waitKey(0);
      
    return 0;
}

Training

For training a larger model, you may refer to link

References

https://github.com/NVIDIA-AI-IOT/trt_pose - Real-time pose estimation (Python and C++)
https://github.com/CaoWGG/TensorRT-YOLOv4 - Object detection based on Tensorrt (C++)

tensorrt-openpose's People

Contributors

Stargazers

Watchers

Forkers

batsukh-msm quant-foundry-limited qingshuang2 zhanghongyong123456 riztazz okinter11 130347665 rexiome dfqytcom rlewkowicz altarizer jinqi2376 linter0663 forexwiki

tensorrt-openpose's Issues

cant understand

https://github.com/spacewalk01/TensorRT-Openpose/blob/248de017b7841b033605a17c3f67c606dceaf402/trt_demo/resize.cu#L44

About attitude tracking real-time accuracy

I was testing the performance and I couldn't keep up with the pose，I can't keep up with the action，I need to adjust those parameters or Ihave a better model, what do I do,I was hoping you could give me some advice，Thank you ever so much

change another model

你好,当我使用从IMU官方的模型从caffe转换到engine 然后使用这个代码发现输出的结果不太理解，而且维度也有点区别
下图是输出信息

but the result very poor this is my input and output

but the result very poor this is my input and output
this is input and out

Originally posted by @chh7411898 in #7 (comment)

Tensorrt context inference error

Hello, when I ran the demo, I found that the context would report an error when inference.
Native environment:
Windows 10
Visual Studio 2019
RTX 2080 Ti GPU
TensorRT 7.2.3.4
CUDA 11.1, Cudnn 8
Python 3.6
Torch 1.8.1
OpenCV 4.5.1 with CUDA

I am getting a crash running this

@spacewalk01 thanks for this great implementation.
I am having some trouble using this repo.

This line in the code does not make sense to me: https://github.com/spacewalk01/TensorRT-Openpose/blob/14714f954875f054f3aab827700f7f1d9f677500/trt_demo/parse/paf_score_graph.cpp#L94
Here counts is a vector of size N*C, C and N are coming from dimensions of input (NCHW) which is 1x3x224x224.
So counts is having a size 3, in the same functions, it's trying to access indexes larger than 2, which is coming from topology,
and that's reason for crash I am getting in https://github.com/spacewalk01/TensorRT-Openpose/blob/14714f954875f054f3aab827700f7f1d9f677500/trt_demo/parse/paf_score_graph.cpp#L24

here b is having garbage as it's coming from counts[4], while counts is having on 3 values.

This is how I am using it:

#include "trt_demo/Openpose.h"
#include "trt_demo/TensorrtPoseNet.h"

#include <string>

int main() {

    TensorrtPoseNet *trt_pose = new TensorrtPoseNet("trt_pose_fp16.engine", 0.9, 0.1);
    std::string filepath="images/image.png";

    cv::Mat img = cv::imread(filepath);

    if (img.rows > 0 and img.cols > 0) {
        cv::resize(img, img, cv::Size(224, 224));
        std::cout << "input image shape:" << img.rows << "x" << img.cols << std::endl;
        trt_pose->infer(img);

        for (int i = 0; i < trt_pose->inputDims.size(); i++) {
            std::cout << "dims: " << trt_pose->inputDims[i].d[0] << " " << trt_pose->inputDims[i].d[1] <<
                    " " << trt_pose->inputDims[i].d[2] << " " << trt_pose->inputDims[i].d[3] << std::endl;
            Openpose *openpose = new Openpose(trt_pose->inputDims[i]);
            openpose->detect(trt_pose->cpuCmapBuffer, trt_pose->cpuPafBuffer, img);
        }
    }
    return 0;
}

I am doing something wrong here? Could please you share a sample-code explaining how to use this?

There was an error getting Engine

My configuration
win10 cuda10.1 cudnn 7.6.5 tensorRT 6.0.1.5 pytorch 1.8.1-cu101 python3.7 VS2019
I wonder if specific CUDA 11.1 and TensorRT are required to build the Engine

How to use another engine in this programme

Hello, I have just run the demo, and it ran smoothly. Now I want to change the engine and to build a new one from the offical model from https://github.com/CMU-Perceptual-Computing-Lab/openpose/tree/master/models/pose/body_25 . I have just transformed the caffe model to engine, but it does not work on this program. The error is : Cuda Error in cuInt8::NCHWToNHWC: 1 (invalid argument) FAILED_EXECUTION: Unknown exception

I wonder is this program only fitted to the resnet model? If so, how can I adapted it to my own?

Detect header says it takes vectors, but the cpp function takes floats

Title. It causes issues at link time. I see folks have made this work,

https://github.com/spacewalk01/tensorrt-openpose/blob/main/trt_demo/Openpose.h#L37
https://github.com/spacewalk01/tensorrt-openpose/blob/main/trt_demo/Openpose.cpp#L3

Looks like folks have built this, but idk how

How to build it using cmake on linux?

I want to test it on ubuntu. I've installed all dependencies, but i don't know how to set the CMakeLists.txt. Could you give an example of it?