Giter Club home page Giter Club logo

open-mmlab / mmpose Goto Github PK

View Code? Open in Web Editor NEW
5.0K 5.0K 1.1K 54.17 MB

OpenMMLab Pose Estimation Toolbox and Benchmark.

Home Page: https://mmpose.readthedocs.io/en/latest/

License: Apache License 2.0

Python 98.77% Shell 0.12% Dockerfile 0.04% C++ 0.69% Jupyter Notebook 0.38%
animal-pose-estimation benchmark cpm crowdpose face-keypoint freihand hand-pose-estimation higher-hrnet hourglass hrnet human-pose mmpose mpii mspn ochuman pose-estimation pytorch rsn rtmpose udp

mmpose's Introduction

Introduction

English | 简体中文

MMPose is an open-source toolbox for pose estimation based on PyTorch. It is a part of the OpenMMLab project.

The main branch works with PyTorch 1.8+.

mmpose.demo.mp4

Major Features
  • Support diverse tasks

    We support a wide spectrum of mainstream pose analysis tasks in current research community, including 2d multi-person human pose estimation, 2d hand pose estimation, 2d face landmark detection, 133 keypoint whole-body human pose estimation, 3d human mesh recovery, fashion landmark detection and animal pose estimation. See Demo for more information.

  • Higher efficiency and higher accuracy

    MMPose implements multiple state-of-the-art (SOTA) deep learning models, including both top-down & bottom-up approaches. We achieve faster training speed and higher accuracy than other popular codebases, such as HRNet. See benchmark.md for more information.

  • Support for various datasets

    The toolbox directly supports multiple popular and representative datasets, COCO, AIC, MPII, MPII-TRB, OCHuman etc. See dataset_zoo for more information.

  • Well designed, tested and documented

    We decompose MMPose into different components and one can easily construct a customized pose estimation framework by combining different modules. We provide detailed documentation and API reference, as well as unittests.

What's New

  • Release RTMO, a state-of-the-art real-time method for multi-person pose estimation.

    rtmo

  • Release RTMW models in various sizes ranging from RTMW-m to RTMW-x. The input sizes include 256x192 and 384x288. This provides flexibility to select the right model for different speed and accuracy requirements.

  • Support inference of PoseAnything. Web demo is available here.

  • Support for two new datasets:

  • Welcome to use the MMPose project. Here, you can discover the latest features and algorithms in MMPose and quickly share your ideas and code implementations with the community. Adding new features to MMPose has become smoother:

    • Provides a simple and fast way to add new algorithms, features, and applications to MMPose.
    • More flexible code structure and style, fewer restrictions, and a shorter code review process.
    • Utilize the powerful capabilities of MMPose in the form of independent projects without being constrained by the code framework.
    • Newly added projects include:
    • Start your journey as an MMPose contributor with a simple example project, and let's build a better MMPose together!

  • January 4, 2024: MMPose v1.3.0 has been officially released, with major updates including:

    • Support for new datasets: ExLPose, H3WB
    • Release of new RTMPose series models: RTMO, RTMW
    • Support for new algorithm PoseAnything
    • Enhanced Inferencer with optional progress bar and improved affinity for one-stage methods

    Please check the complete release notes for more details on the updates brought by MMPose v1.3.0!

0.x / 1.x Migration

MMPose v1.0.0 is a major update, including many API and config file changes. Currently, a part of the algorithms have been migrated to v1.0.0, and the remaining algorithms will be completed in subsequent versions. We will show the migration progress in this Roadmap.

If your algorithm has not been migrated, you can continue to use the 0.x branch and old documentation.

Installation

Please refer to installation.md for more detailed installation and dataset preparation.

Getting Started

We provided a series of tutorials about the basic usage of MMPose for new users:

  1. For the basic usage of MMPose:

  2. For developers who wish to develop based on MMPose:

  3. For researchers and developers who are willing to contribute to MMPose:

  4. For some common issues, we provide a FAQ list:

Model Zoo

Results and models are available in the README.md of each method's config directory. A summary can be found in the Model Zoo page.

Supported algorithms:
Supported techniques:
Supported datasets:
Supported backbones:

Model Request

We will keep up with the latest progress of the community, and support more popular algorithms and frameworks. If you have any feature requests, please feel free to leave a comment in MMPose Roadmap.

Contributing

We appreciate all contributions to improve MMPose. Please refer to CONTRIBUTING.md for the contributing guideline.

Acknowledgement

MMPose is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their own new models.

Citation

If you find this project useful in your research, please consider cite:

@misc{mmpose2020,
    title={OpenMMLab Pose Estimation Toolbox and Benchmark},
    author={MMPose Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmpose}},
    year={2020}
}

License

This project is released under the Apache 2.0 license.

Projects in OpenMMLab

  • MMEngine: OpenMMLab foundational library for training deep learning models.
  • MMCV: OpenMMLab foundational library for computer vision.
  • MMPreTrain: OpenMMLab pre-training toolbox and benchmark.
  • MMagic: OpenMMLab Advanced, Generative and Intelligent Creation toolbox.
  • MMDetection: OpenMMLab detection toolbox and benchmark.
  • MMDetection3D: OpenMMLab's next-generation platform for general 3D object detection.
  • MMRotate: OpenMMLab rotated object detection toolbox and benchmark.
  • MMTracking: OpenMMLab video perception toolbox and benchmark.
  • MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark.
  • MMOCR: OpenMMLab text detection, recognition, and understanding toolbox.
  • MMPose: OpenMMLab pose estimation toolbox and benchmark.
  • MMHuman3D: OpenMMLab 3D human parametric model toolbox and benchmark.
  • MMFewShot: OpenMMLab fewshot learning toolbox and benchmark.
  • MMAction2: OpenMMLab's next-generation action understanding toolbox and benchmark.
  • MMFlow: OpenMMLab optical flow toolbox and benchmark.
  • MMDeploy: OpenMMLab Model Deployment Framework.
  • MMRazor: OpenMMLab model compression toolbox and benchmark.
  • MIM: MIM installs OpenMMLab packages.
  • Playground: A central hub for gathering and showcasing amazing projects built upon OpenMMLab.

mmpose's People

Contributors

ailingzengzzz avatar ben-louis avatar bladrome avatar canwang-sjtu avatar cherryjm avatar dominic23331 avatar dreampoet avatar hobeom avatar innerlee avatar jin-s13 avatar joannalxy avatar juxuan27 avatar lareinam avatar liqikai9 avatar liuxin9608 avatar luminxu avatar ly015 avatar motokimura avatar rangilyu avatar tau-j avatar tommyzihao avatar vansin avatar wusize avatar xin-li-67 avatar xinxinxinxu avatar yaochaorui avatar zengwang430521 avatar zheng-linxiao avatar zhiyuan-ren avatar zwfcrazy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mmpose's Issues

Iteration Plan - Sep 2020

Version 0.6 will be released soon. Now it's time to plan for the next iteration, i.e. v0.7. This will be a one-month iteration.

Algorithms:

Datasets:

Tools:

Roadmap of MMPose

We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan is also available here.

You can either:

  1. Suggest a new feature by leaving a comment.
  2. Vote for a feature request with 👍 or be against with 👎. (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!)
  3. Tell us that you would like to help implement one of the features in the list or review the PRs. (This is the greatest things to hear about!)

Realtime performance question

Hello and first of all thanks for the great tool you have built.

I wanted to ask some questions, I run the pretrained model on a 1080 ti on a realtime video and obtain so-so performances :
res50_coco_256x192: 12 FPS
mobilenetv2_coco_256x192: 15 FPS

Am I missing something or is it just that the GPU isn't strong enough. Thanks!

Slow inference of bottom-up models

I am getting very slow performance on a single person video on RTX2080Ti with the following command

python demo/bottom_up_video_demo.py configs/bottom_up/resnet/coco/res152_coco_512x512.py res152_coco_512x512-364eb38d_20200822.pth --video-path demo/tabata1_640x360.mp4 --show --device cuda:0

FPS ~2.5. It was calculated as total processing time divided by the number of video frames, so it is not super precise...

Any idea what might be wrong?

[help wanted] console log freezes while train...

image
it has cost almost 1 hours ...and it seems that it doesn't train...
does any body meet this case before? many thanks..........


log_level = 'INFO'
load_from = None
resume_from = None
dist_params = dict(backend='nccl')
workflow = [('train', 1)]
checkpoint_config = dict(interval=10)
evaluation = dict(interval=10, metric='mAP')

optimizer = dict(
    type='Adam',
    lr=5e-4,
)
optimizer_config = dict(grad_clip=None)
# learning policy
lr_config = dict(
    policy='step',
    warmup='linear',
    warmup_iters=500,
    warmup_ratio=0.001,
    step=[170, 200])
total_epochs = 210
log_config = dict(
    interval=50,
    hooks=[
        dict(type='TextLoggerHook'),
        # dict(type='TensorboardLoggerHook')
    ])

channel_cfg = dict(
    num_output_channels=17,
    dataset_joints=17,
    dataset_channel=[
        [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16],
    ],
    inference_channel=[
        0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16
    ])

# model settings
model = dict(
    type='TopDown',
    pretrained='models/pytorch/imagenet/'
    'mobilenet_v2_batch256_20200708-3b2dc3af.pth',
    backbone=dict(type='MobileNetV2', widen_factor=1., out_indices=(7, )),
    keypoint_head=dict(
        type='TopDownSimpleHead',
        in_channels=1280,
        out_channels=channel_cfg['num_output_channels'],
    ),
    train_cfg=dict(),
    test_cfg=dict(
        flip_test=True,
        post_process=True,
        shift_heatmap=True,
        unbiased_decoding=False,
        modulate_kernel=11),
    loss_pose=dict(type='JointsMSELoss', use_target_weight=True))

data_cfg = dict(
    image_size=[192, 256],
    heatmap_size=[48, 64],
    num_output_channels=channel_cfg['num_output_channels'],
    num_joints=channel_cfg['dataset_joints'],
    dataset_channel=channel_cfg['dataset_channel'],
    inference_channel=channel_cfg['inference_channel'],
    soft_nms=False,
    nms_thr=1.0,
    oks_thr=0.9,
    vis_thr=0.2,
    bbox_thr=1.0,
    use_gt_bbox=False,
    image_thr=0.0,
    bbox_file='data/coco/person_detection_results/'
    'COCO_val2017_detections_AP_H_56_person.json',
)

train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='TopDownRandomFlip', flip_prob=0.5),
    dict(
        type='TopDownHalfBodyTransform',
        num_joints_half_body=8,
        prob_half_body=0.3),
    dict(
        type='TopDownGetRandomScaleRotation', rot_factor=40, scale_factor=0.5),
    dict(type='TopDownAffine'),
    dict(type='ToTensor'),
    dict(
        type='NormalizeTensor',
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]),
    dict(type='TopDownGenerateTarget', sigma=2),
    dict(
        type='Collect',
        keys=['img', 'target', 'target_weight'],
        meta_keys=[
            'image_file', 'joints_3d', 'joints_3d_visible', 'center', 'scale',
            'rotation', 'bbox_score', 'flip_pairs'
        ]),
]

valid_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='TopDownAffine'),
    dict(type='ToTensor'),
    dict(
        type='NormalizeTensor',
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]),
    dict(
        type='Collect',
        keys=[
            'img',
        ],
        meta_keys=[
            'image_file', 'center', 'scale', 'rotation', 'bbox_score',
            'flip_pairs'
        ]),
]

test_pipeline = valid_pipeline

data_root = '/share/jwh/utils/data/zipped/object/detect/coco/coco2017'
data = dict(
    samples_per_gpu=64,
    workers_per_gpu=2,
    train=dict(
        type='TopDownCocoDataset',
        ann_file=f'{data_root}/annotations/person_keypoints_val2017.json',
        img_prefix=f'{data_root}/val2017/',
        data_cfg=data_cfg,
        pipeline=train_pipeline),
    val=dict(
        type='TopDownCocoDataset',
        ann_file=f'{data_root}/annotations/person_keypoints_val2017.json',
        img_prefix=f'{data_root}/val2017/',
        data_cfg=data_cfg,
        pipeline=valid_pipeline),
    test=dict(
        type='TopDownCocoDataset',
        ann_file=f'{data_root}/annotations/person_keypoints_val2017.json',
        img_prefix=f'{data_root}/val2017/',
        data_cfg=data_cfg,
        pipeline=valid_pipeline),
)

Iteration Plan - Sep 2020

Version 0.6 will be released soon. Now it's time to plan for the next iteration, i.e. v0.7. This will be a one-month iteration and our focus in this iteration is to enrich the supported tasks. We will add support for 3D shapes, and add more hand tasks.

Algorithms:

  • HMR (3D human shape recovery) (#162)

Datasets:

  • LSP (not needed any more)
  • H36M (#159)
  • COCO-WholeBody (#78)
  • Frei-hand (#125)
  • Panotic (#144)

Tools:

Misc:

batch inference

Hello,
I am trying to find a way to do batch inference with mmpose, but it seems that the project has not supported it yet. There is only a function called "_inference_single_pose_model" inside inference.py.

Is there any quick way I can adapt mmpose to batch inference? I think it's an essential feature because most of the time, we just work on bach inference. real-time infernece with a single input image is not possible with the current architectures, so single-image inference is kind of not useful.

Thank you and I look forward to hearing from you soon.
Best.

cannot achieve reported training speed

Hi, I can not achieve the reported training speed for multiple configs.

The followings are my test.

Model Input size Batch size per gpu MMPose (s/iter)
Resnet_50 256x192 64 0.88
Resnet_50 384x288 64 1.3
Resnet_101 256x192 64 0.88
Resnet_101 384x288 32 0.64
Hrnet_w32 256x192 64 0.88

4 NVidia V100SXM2 (16G memory)
2 x Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz

inference speed is low

I simply tested the speed.
GPU: T4
There are two people in the test video.

mmpose:
res50_coco_256x192: 27 FPS
mobilenetv2_coco_256x192: 31 FPS

alphapose:
fast_res50_256x192: 50 FPS

Why is there such a big gap in keypoint_head.

Support Demoing HigherHRNet

Hi,

Thank you for your interesting work. I am trying to test HigherHRNet on a collection of videos I've found very challenging for existing human pose estimation models. However when I try to execute the following command:

python demo/video_demo_with_mmdet.py $SCRATCH/mmdetection/configs/faster_rcnn/faster_rcnn_r50_fpn_1x_coco.py $SCRATCH/mmdetection/checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth configs/bottom_up/higherhrnet/coco/higher_hrnet48_coco_512x512.py checkpoints/higher_hrnet48_coco_512x512-60fedcbc_20200712.pth --video-path $SCRATCH/VIBE/sample_video.mp4 --show --out-video-root ./

I get a number of config related issues, the first is that the image size seems to be treated as both an integer as well as a list, I made a temporary fix with a try except statement, but the second issue I'm dealing with now is the lack of a flip_index.

I am not sure what purpose the flip_index has but I've tried making it 0 or [0,1] as I saw in another example and neither of these work.

For a detailed traceback I get

Traceback (most recent call last):
  File "demo/video_demo_with_mmdet.py", line 117, in <module>
    main()
  File "demo/video_demo_with_mmdet.py", line 90, in main
    format='xyxy')
  File "/mmpose/mmpose/apis/inference.py", line 238, in inference_pose_model
    pose = _inference_single_pose_model(model, img_or_path, bbox)
  File "mmpose/mmpose/apis/inference.py", line 194, in _inference_single_pose_model
    return_loss=False, img=data['img'], img_metas=data['img_metas'])
  File ".conda/envs/pytorch150/lib/python3.7/site-packages/torch/nn/modules/module.py", line 550, in __call__
    result = self.forward(*input, **kwargs)
  File "mmpose/mmpose/models/detectors/bottom_up.py", line 103, in forward
    return self.forward_test(img, img_metas, **kwargs)
  File "mmpose/mmpose/models/detectors/bottom_up.py", line 218, in forward_test
    self.test_cfg['flip_test'])
  File "mmpose/mmpose/core/evaluation/bottom_up_eval.py", line 157, in aggregate_results
    heatmaps[1]) / 2.0 if flip_test else heatmaps[0]

Hope you can help me, I'm really excited about your work and hoping to be able to make contributions in the future.

Inference script for bottom_up model

Hello guys.
Thank you for your great contribution.

I tried some top-down method (model zoo by using the scripts inside mmpose/demo, and it works well.

Unfortunately, I can't run the bottom-up model yet.
Do you guys have any clue how to do this?

Thank you.

Question about validate during training, about valid_pipeline and bbox_file

hi,
thx for your great work,

Im using configs/top_down/resnet/coco/res50_coco_256x192.py to train my own keypoint dataset(git mmpose at 2020.08.06), theres some questions i met:

[line72--73] have been changed to my own file:

bbox_file='data/person_detection_results/'
'bbox_1000pic.json',

[line126--137] changed to:

    train=dict(
    type='TopDownCocoDataset',
    ann_file=f'{data_root}/annotations/person_keypoints_train_1000pic.json',
    img_prefix=f'{data_root}/train_1000pic/',
    data_cfg=data_cfg,
    pipeline=train_pipeline),

    val=dict(
    type='TopDownCocoDataset',
    ann_file=f'{data_root}/annotations/person_keypoints_train_1000pic.json',
    img_prefix=f'{data_root}/train_1000pic/',
    data_cfg=data_cfg,
    pipeline=valid_pipeline),

In this setting, everything is alright, train on 1000pic successfully, valid on 1000pic successfully, output result.json in work_dir successfully;


but when i change valid, for example, when valid part changed from:

    ann_file=f'{data_root}/annotations/person_keypoints_train_1000pic.json',
    img_prefix=f'{data_root}/train_1000pic/',

to

    ann_file=f'{data_root}/annotations/person_keypoints_valid_20pic.json'
    img_prefix=f'{data_root}/valid_20pic/',

it seems mmpose read bbox_1000pic.json from line72--73, and consider there should be 1000images to valid, then read valid dir from line135 to find it;

In other word, on my understanding, mmpose use image_id from bbox_1000pic.json which read at line72--73(for example image_id:21, btw, person_keypoints_valid_20pic.json only has 20images, doesnot has image_id:21), then, try to find it in valid dir
(try to find 000 000 000 021.jpg in {data_root}/valid_21pic/). Error showed as: FileNotFoundError: img file does not exist: data/valid_20pic/000000000021.jpg

So~ config file can be only put 1 train_1000_bbox.json in it, but there are 2 ,train_1000.json and valid_20.json, in train_pipeline,valid_pipeline,
Are there any way to change config file to provide valid_20_bbox.json for valid_pipeline, or teach it to read bbox info from valid_20.json.


Thankyou very much.

onehand10k vis results?

Looking at the viz_pose_results funciton used in topdown image demo it looks like this is specifically for 17 keypoint pose estimation. Is there another function that needs to be used to visualize hand poses or is this not implemented yet?

How to save model when training?

So I'm trying to learn how to use this repo, but I can't figure out how to save the model. I assumed it would be saved in the same directory as the logs are created but don't see any .pth files there.

I've done the following when training a resnet50 model on the mpii dataset on 2 gpus

./tools/dist_train.sh configs/top_down/resnet/mpii/res50_mpii_256x256_test.py 2

I have a pretrained model at models/pytorch/imagenet/resnet50-19c8e357.pth

I train for 2 epochs and get the .log and .log.json files + a pred.mat though I'm not sure what the .mat format is?
Where does the .pth file save? I've tried digging into the code but haven't really found where and it seems like currently, I can only finetune to save a model? does it only save based on a certain eval metric (PCKh/mAP)?

Any help would be appreciated, thanks!

Specify the minimum version of mmcv

For now, we only restrict the minimum version of mmcv.

Restricting upper bound sounds attractive but it might invites trouble when you want to use multiple repos in mm-series.

performance question

I followed get_start.md , download model form https://mmpose.readthedocs.io/en/latest/model_zoo.html model_name is pose_resnet_50 and img_size is 156*192
use
python -m torch.distributed.launch --nproc_per_node=8 tools/test.py configs/top_down/resnet/coco/res50_coco_256x192.py checkpoints/res50_coco_256x192-ec54d7f3_20200709.pth --launcher pytorch --eval mAP

but got
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.735
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.925
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.814
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.706
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.779
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.765
Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.934
Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.833
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.733
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.814
so whats wrong?

wrong cfg?

tried to run demo
image
image size in config file is single integer but in _box2cs() expecting sequantial
image
I use higher_hrnet32_coco_512x512.py config
image

MPII dataset processing

It's a really fantastic work! Can you share the processing code on how to transform the .mat file to json file? I've found the annotations is in the type of .mat file on the MPII Dataset website. Thanks a lot!!

unexpected keys in pre-trained model loading

Hello, thank you very much for creating a very helpful pose estimation project.
I am trying to fine-tune the model configs/top_down/hrnet/coco/hrnet_w48_coco_384x288.py with the pre-trained weights: hrnet_w48_coco_384x288-314c8528_20200708.pth on the posetrack dataset, and the below warning show up the console.

The training command line I used is:
python tools/train.py ./configs/top_down/hrnet_mvai/coco/hrnet_w48_coco_384x288.py

And here is the pre-trained weight that I used.

And here is the warning message:

unexpected key in source state_dict: backbone.conv1.weight, backbone.bn1.weight, backbone.bn1.bias, backbone.bn1.running_mean, backbone.bn1.running_var, backbone.bn1.num_batches_tracked, backbone.conv2.weight, backbone.bn2.weight, backbone.bn2.bias, backbone.bn2.running_mean, backbone.bn2.running_var, backbone.bn2.num_batches_tracked, backbone.layer1.0.conv1.weight, backbone.layer1.0.bn1.weight, etc

missing keys in source state_dict: conv1.weight, bn1.weight, bn1.bias, bn1.running_mean, bn1.running_var, conv2.weight, bn2.weight, bn2.bias, bn2.running_mean, bn2.running_var, layer1.0.conv1.weight, layer1.0.bn1.weight, layer1.0.bn1.bias, layer1.0.bn1.running_mean, layer1.0.bn1.running_var, layer1.0.conv2.weight, layer1.0.bn2.weight, layer1.0.bn2.bias, layer1.0.bn2.running_mean, layer1.0.bn2.running_var, layer1.0.conv3.weight, layer1.0.bn3.weight, layer1.0.bn3.bias, layer1.0.bn3.running_mean, layer1.0.bn3.running_var, layer1.0.downsample.0.weight, layer1.0.downsample.1.weight, layer1.0.downsample.1.bias, layer1.0.downsample.1.running_mean, layer1.0.downsample.1.running_var, layer1.1.conv1.weight, layer1.1.bn1.weight, layer1.1.bn1.bias, layer1.1.bn1.running_mean, layer1.1.bn1.running_var, layer1.1.conv2.weight, layer1.1.bn2.weight, layer1.1.bn2.bias, layer1.1.bn2.running_mean, layer1.1.bn2.running_var, layer1.1.conv3.weight, layer1.1.bn3.weight, etc

Multi-GPU training return cuda memory error

Pardon me, buddies, I'm planed to training the wonderful frame, and used this command: ./tools/dist_train.sh configs/top_down/resnet/coco/res50_coco_256x192.py 2 --no-validate;
So, I got a cuda memory problem: when I set gpu_num=2, it's return a cuda memory error. If I changed gpu_num =1, it's works, and begin training, no memory error return.
The Nvidia configuration (set gpu=1, it has been running):
| NVIDIA-SMI 440.36 Driver Version: 440.36 CUDA Version: 10.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce RTX 2070 Off | 00000000:01:00.0 On | N/A |
| 90% 71C P2 170W / 175W | 7966MiB / 7979MiB | 97% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce RTX 2070 Off | 00000000:03:00.0 Off | N/A |
| 50% 40C P8 9W / 175W | 12MiB / 7982MiB | 0% Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage

Did my torch envs problem? Or system problem?

bbox score must be saved as float in coco json file

Thanks for the extraordinary work!

As I am trying to test on my own data, I have found that if the bounding box scores in coco format json is saved as 1 (instead of 1.0), there will be the info => Total boxes after filter low [email protected]: 0 and test does not start. I changed the scores to 0.999 and it works.

Is this deliberate, or is it inherited from pycocotools?

Issue of generating COCO_val2017_detections_AP_H_56_person.json

2020-08-24 19-18-25屏幕截图
Firstly, thanks for sharing the mmpose as open source project.

The code is successfully installed on my PC and the demo runs successfully. However, when I try to run scripts/test.py in order to check the mAP on mscoco dataset, the error happened with FileNotFoundError: [Errno 2] No such file or directory: 'data/coco/person_detection_results/COCO_val2017_detections_AP_H_56_person.json'. I thought when I the .json file should be generated when the demo run. Am I right ? Is there any suggestion that how can I find or generate this .json file ?

Thank you and Best wishes

config issue

configs/top_down/resnet/coco/res50_coco_256x192.py
missed
test_pipeline = valid_pipeline

mmcv toolkits can not import

Hi, buddies, In my virtual env, I run a test demo: ./tools/dist_test.sh configs/top_down/resnet/coco/res50_coco_256x192.py checkpoints/res50_coco_256x192-ec54d7f3_20200709.pth 2 --eval mAP
and I got a env problems: ImportError: cannot import name 'EpochBasedRunner' from 'mmcv.runner' (/home/stelo/anaconda3/envs/mmpose/lib/python3.7/site-packages/mmcv/runner/init.py)
should I rebuild my env? Or conda remove mmcv packages and reinstall it?

Issue with testing using distributed and mpii

So I was trying to test the mpii dataset using both test.py and ./tool/dist_test.sh and came across this small issue. It looks like mpii uses the evaluation metric PCKh rather than mPA so I changed the args as follows

For test.py

 python tools/test.py configs/top_down/hrnet/mpii/hrnet_w32_mpii_256x256.py ~/0Dev/data/models/hrnet_w32_mpii_256x256-6c4f923f_20200812.pth --eval PCKh

for ./tools/dist_test.sh

./tools/dist_test.sh configs/top_down/hrnet/mpii/hrnet_w32_mpii_256x256.py ~/0Dev/data/models/hrnet_w32_mpii_256x256-6c4f923f_20200812.pth 2 --eval PCKh

I get the assertion error saying that metric != PCKh and modified the error so I could figure out what was going on and saw

assert metric == 'PCKh', f"got {metric}"
AssertionError: got ['PCKh']

I'm fairly new to this repo so I don't know the ins and outs enough to make a pull request, so I wanted to make you guys aware of this issue!

Git clone git clone [email protected]:open-mmlab/mmpose.git doesn't work

Don't know if it is just me. Cloning the repository form the mentioned gitlab link doesn't work. The connection times out.

(HRNet) akilesh@heracleiapc:~/PyCharmProjects$ git clone [email protected]:open-mmlab/mmpose.git
Cloning into 'mmpose'...
ssh: connect to host gitlab.sz.sensetime.com port 22: Connection timed out
fatal: Could not read from remote repository.

Please make sure you have the correct access rightsand the repository exists.

I cloned with the github link though.!

MPII evaluation

hello, I've read the evaluation code for MPII dataset, it calculated about the average accracy for all the keypoints. However, on its official website, the evaluation has been divided into two parts: single-person and multi-person, a little bit different from this repo. So does this repo use the same evaluaion tools as the official realization or only calculate the averaged results for all keypoints in all images? Hope for your reply! Thanks a lot!!

Temporal video analysis

Does MMPose support a temporal analysis of videos yet, where the pose estimation is not done on a frame-by-frame basis, but rather using temporal information, like in this paper on VideoPose3D ?
Supporting temporal frameworks would need the datasets to consist of an unordered set of videos, where each video is a sequential ordered list of frames. Furthermore, the splitting of training and validation set, the definition of batches and the input to the model would also need to work accordingly.

benchmark

A very nice work!

I have some questions about the benchmark.

  1. In the benchmark table, I found that with the same input size, all the models of res50, 101, 152, hrnet-w32, and hrnet-w48 have almost the same speed. However, all these models have different GFLOPs. Could you explain why this happens?
  2. Comparing with the implementation of HRNet, the training speed of hrnet serious backbone has a big improvement. Could you provide more details about where the improvement comes from?

Another suggestion about the benchmark. Maybe you can consider also provide the inference speed.

AttributeError: 'ConfigDict' object has no attribute 'test_pipeline'

excuse me, there's some error when i try to use demo/img_demo.py
would you please give me some advices, thank you very very much.
truly sorry for wasting your time.

  1. use hrnet_w32_coco_384x288.py
    + hrnet_w32_coco_384x288-d9f0d786_20200708.pth
    on coco2017val

(mmpose) :/work/wjw_2020/app/mmpose-master_20200716/mmpose$ python demo/img_demo.py
configs/top_down/hrnet/coco/hrnet_w32_coco_384x288.py
work/hrnet_w32_coco_384x288-d9f0d786_20200708.pth
--img-root data/coco/val2017/
--json-file data/coco/annotations/person_keypoints_val2017.json
--show

loading annotations into memory...
Done (t=0.24s)
creating index...
index created!
Traceback (most recent call last):
File "demo/img_demo.py", line 91, in
main()
File "demo/img_demo.py", line 72, in main
pose_model, image_name, person_bboxes, format='xywh')
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmpose/apis/inference.py", line 230, in inference_pose_model
pose = _inference_single_pose_model(model, img_or_path, bbox)
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmpose/apis/inference.py", line 151, in _inference_single_pose_model
test_pipeline = [LoadImage()] + cfg.test_pipeline[1:]
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmcv/mmcv/utils/config.py", line 349, in getattr
return getattr(self._cfg_dict, name)
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmcv/mmcv/utils/config.py", line 37, in getattr
raise ex
AttributeError: 'ConfigDict' object has no attribute 'test_pipeline'


  1. use hrnet_w32_coco_384x288.py
    + hrnet_w32_coco_384x288-d9f0d786_20200708.pth
    on test folder

(mmpose):/work/wjw_2020/app/mmpose-master_20200716/mmpose$ python demo/img_demo.py \
configs/top_down/hrnet/coco/hrnet_w32_coco_384x288.py
work/hrnet_w32_coco_384x288-d9f0d786_20200708.pth
--img-root tests/data/
--json-file tests/data/test_coco.json
--show

loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Traceback (most recent call last):
File "demo/img_demo.py", line 91, in
main()
File "demo/img_demo.py", line 72, in main
pose_model, image_name, person_bboxes, format='xywh')
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmpose/apis/inference.py", line 230, in inference_pose_model
pose = _inference_single_pose_model(model, img_or_path, bbox)
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmpose/apis/inference.py", line 151, in _inference_single_pose_model
test_pipeline = [LoadImage()] + cfg.test_pipeline[1:]
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmcv/mmcv/utils/config.py", line 349, in getattr
return getattr(self._cfg_dict, name)
File "/home/gy/work/wjw_2020/app/mmpose-master_20200716/mmpose/mmcv/mmcv/utils/config.py", line 37, in getattr
raise ex
AttributeError: 'ConfigDict' object has no attribute 'test_pipeline'


3-1.configs/bottom_up/higherhrnet/coco/higher_hrnet32_coco_512x512.py
+ higher_hrnet32_coco_512x512-8ae85183_20200713.pth
same error: AttributeError: 'ConfigDict' object has no attribute 'test_pipeline'

3-2.configs/top_down/resnet/coco
+ res50_coco_256x192-ec54d7f3_20200709.pth
same error: AttributeError: 'ConfigDict' object has no attribute 'test_pipeline'

4.other infomations:
4-1.ubuntu16.04, 1080Ti, Python 3.7.7, pytorch 1.5.1, GCC 7.3.0, mmcv 1.0.2, numpy 1.16.4, opencv 4.3.0, json_tricks 3.15.2
4-2.CUDA Version: 10.1 Driver Version: 418.87.00
4-3.get mmpose at 2020.07.16 by: git clone https://github.com/open-mmlab/mmpose.git
4-4.get mmcv at 2020.07.16 by: pip install mmcv
same error after try to uninstall and 'pip install mmcv-full';
same error after try to uninstall and 'pip install mmcv-full==latest+torch1.5.0+cu101 -f https://openmmlab.oss-accelerate.aliyuncs.com/mmcv/dist/index.html' command in https://github.com/open-mmlab/mmcv,
same error after try to uninstall and 'git clone https://github.com/open-mmlab/mmcv.git && cd mmcv && pip install -e .'
same error after try to uninstall and 'git clone https://github.com/open-mmlab/mmcv.git && cd mmcv && MMCV_WITH_OPS=1 pip install -e .'

4-6.conda list (it's too long, sorry!):
Name Version Build Channel
_libgcc_mutex 0.1 main
addict 2.2.1
albumentations 0.4.6
appdirs 1.4.4
blas 1.0 mkl
ca-certificates 2020.6.24 0
certifi 2020.6.20 py37_0
cityscapesScripts 1.5.0
clang-format 9.0.0
cudatoolkit 10.1.243 h6bb024c_0
cycler 0.10.0
Cython 0.29.21
decorator 4.4.2
freetype 2.10.2 h5ab3b9f_0
future 0.18.2
imagecorruptions 1.1.0
imageio 2.9.0
imgaug 0.4.0
intel-openmp 2020.1 217
jpeg 9b 0
json-tricks 3.15.2
kiwisolver 1.2.0
ld_impl_linux-64 2.33.1 h53a641e_7
libedit 3.1.20191231 h7b6447c_0
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libpng 1.6.37 hbc83047_0
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
lz4-c 1.9.2 he6710b0_0
matplotlib 3.2.2
mkl 2020.1 217
mkl_fft 1.0.12 py37ha843d7b_0
mkl_random 1.1.1 py37h0573a6f_0
mmcv 1.0.2
mmcv-full 1.0.2
mmcv-nightly 1.0rc0.dev20200630
mmdet 2.3.0rc0+5f2cded
mmpose 0.1.0+unknown
munkres 1.0.7 py_1
ncurses 6.2 he6710b0_1
networkx 2.4
ninja 1.7.2 0
numpy 1.16.4 py37h7e9f1db_0
numpy-base 1.16.4 py37hde5b4d6_0
olefile 0.46 py_0
opencv-python 4.3.0.36
openssl 1.1.1g h7b6447c_0
Pillow 6.2.2
pillow 7.1.2 py37hb39fc2d_0
pip 20.1.1 py37_1
pycocotools 2.0
pyparsing 2.4.7
python 3.7.7 hcff3b4d_5
python-dateutil 2.8.1
pytorch 1.5.1 py3.7_cuda10.1.243_cudnn7.6.3_0 pytorch
PyWavelets 1.1.1
PyYAML 5.3.1
readline 8.0 h7b6447c_0
scikit-image 0.17.2
scipy 1.5.1
setuptools 47.3.1 py37_0
Shapely 1.7.0
six 1.15.0
sqlite 3.32.3 h62c20be_0
terminaltables 3.1.0
tifffile 2020.7.4
tk 8.6.10 hbc83047_0
torchvision 0.6.1 py37_cu101 pytorch
wheel 0.34.2 py37_0
xz 5.2.5 h7b6447c_0
yapf 0.30.0
zlib 1.2.11 0
zstd 1.4.4 h0b5b093_3

4-7.one of the config i use,P.S.: have not do any change:
hrnet_w32_coco_384x288.py.txt

4-8.one of the weights i use,
hrnet_w32_coco_384x288-d9f0d786_20200708.pth from https://mmpose.readthedocs.io/en/latest/model_zoo.html
tp1
md5: 105a8076e55599d9e7ab0026eb894326

4-9.other infomation:
this enviroment running mmdet normally/ok;
i have search 'AttributeError: 'ConfigDict' object has no attribute 'test_pipeline'' with bing; then found 2 issue in mmdetection's issues, it said that the issue-asker's config is too old and git lastest mmdet can solve his problem.


maybe i did some wrong operation on enviroment...
look forward to reply,thankyou very much

KeyError: 'rotation'

where can i download this checkpoint。

and i use hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth
but i get the error:
File "demo/top_down_video_demo_with_mmdet.py", line 118, in
main()
File "demo/top_down_video_demo_with_mmdet.py", line 91, in main
format='xyxy')
File "/home/cl/gitclone/mmpose/mmpose/apis/inference.py", line 228, in inference_pose_model
pose = _inference_single_pose_model(model, img_or_path, bbox)
File "/home/cl/gitclone/mmpose/mmpose/apis/inference.py", line 175, in _inference_single_pose_model
data = test_pipeline(data)
File "/home/cl/gitclone/mmpose/mmpose/datasets/pipelines/shared_transform.py", line 78, in call
data = t(data)
File "/home/cl/gitclone/mmpose/mmpose/datasets/pipelines/top_down_transform.py", line 181, in call
r = results['rotation']
KeyError: 'rotation'

Multiple samples per GPU while testing

Hi everybody

Currently only one sample per GPU is possible while executing tests, which leads to very slow validation and testing for me. I assume to fix this the forward_test methods for the TopDown and BottomUp classes would need to be updated. Is this planned anytime soon or should I give it a try myself to contribute?

How to show pose label in result image?

How to show pose label like this.

image

And If it can, Can I train pose? not object.
For example, MMdetction will show three person labels in the above image.
But I want to show another three labels in the above image like sitting, hitting and standing.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.