kimyoon-young / centernet-deep-sort Goto Github PK

Hello, I want to track multiple types of targets, and label each type of target as person 1 person 2 car 1 car2, how can I change this?

I have tried performing cell tracking.
In some cases, the tracker failed (wrongly assigned object ID).
It also sometimes makes new IDs for the reappeared object.
What I need to know is how to evaluate the accuracy of tracking so that we can compare this algorithm with other algorithms?

Any help would be appreciated.

This work only detect the person

==================================== ERRORS ====================================
__________________________ ERROR collecting train.py ___________________________
train.py:44: in
torchvision.datasets.ImageFolder(train_dir, transform=transform_train),
../../anaconda3/envs/CenterNet/lib/python3.6/site-packages/torchvision/datasets/folder.py:178: in init
target_transform=target_transform)
../../anaconda3/envs/CenterNet/lib/python3.6/site-packages/torchvision/datasets/folder.py:79: in init
"Supported extensions are: " + ",".join(extensions)))
E RuntimeError: Found 0 files in subfolders of: /home/ying/centerNet-deep-sort-master/deep/data/train
E Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.51 seconds ============================
Process finished with exit code 0

Why do I change the color of the drawing, always reverse color, input the yellow rgb configuration output is blue

    def _summarizeDets():
        stats = np.zeros((12,))
        stats[0] = _summarize(1)
        stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])
        stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])
        stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])
        stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])
        stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])
        stats[6] = _summarize(0, maxDets=self.params.maxDets[0])
        stats[7] = _summarize(0, maxDets=self.params.maxDets[1])
        stats[8] = _summarize(0, maxDets=self.params.maxDets[2])
        stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])
        stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])
        stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])
        return stats

hi?
i don't know area
what is mean this?
and i want know mAP.
is mAP 'Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ]' ?

ModuleNotFoundError: No module named '_init_paths',plz guide me,thx

JUST RUN COCOEVAL.PY???

ModuleNotFoundError: No module named 'models.decode'

I think you need to convert BGR to RGB before feeding the network. Here:

Where should I change?Thanks.

Hello,
I am trying to work with coco pretrained model+deep sort and i am getting below error:

centerNet-deep-sort/deep/feature_extractor.py", line 12, in init
state_dict = torch.load(model_path)['net_dict']
KeyError: 'net_dict'

Thank you.
Anshu

detections = [Detection(bbox_xywh[i], conf, features[i]) for i,conf in enumerate(confidences)]
TypeError: 'NoneType' object is not iterable

hi,ctdet_coco_dla_2x.pth whether can to onnx or tensorrt?thanks

Hello @kimyoon-young
I was trying your project on my machine, but it doesn't have GPU so I have to reconfig to be able to run on CPU. But i have no idea to do this.
Could you support me, @kimyoon-young ?
Thanks in advance.

Hi young, This code is very good. But I cannot find where is the deep.feature_extractor. So I cannot run the demo. If it is possible, please tell me how to find the deep module. Thanks.

Which dataset can I use to detect cars? How can it be done?

I have change TASK = 'ctdet' # or 'multi_pose' for human pose estimation to multi_pose and download multi_pose_dla_3x.pth from centernet, when i run demo the following occur
<module 'sys' (built-in)>
Fix size testing.
training chunk_sizes: [32]
The output will be saved to /home/pc/fjj_workspace/From_win10/centerNet-deep-sort-master/CenterNet/src/lib/../../exp/multi_pose/default
heads {'hm': 1, 'wh': 2, 'hps': 34, 'reg': 2, 'hm_hp': 17, 'hp_offset': 2}
/home/pc/anaconda3/envs/CenterNet/lib/python3.6/site-packages/sklearn/utils/linear_assignment_.py:21: DeprecationWarning: The linear_assignment_ module is deprecated in 0.21 and will be removed from 0.23. Use scipy.optimize.linear_sum_assignment instead.
DeprecationWarning)
Creating model...
loaded /home/pc/fjj_workspace/From_win10/centerNet-deep-sort-master/CenterNet/models/multi_pose_dla_3x.pth, epoch 320
Drop parameter base.fc.weight.
Drop parameter base.fc.bias.
Loading weights from deep/checkpoint/ckpt.t7... Done!
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument
Traceback (most recent call last):
File "demo_centernet_deepsort.py", line 173, in
det.detect()
File "demo_centernet_deepsort.py", line 132, in detect
bbox_xywh, cls_conf = bbox_to_xywh_cls_conf(results)
File "demo_centernet_deepsort.py", line 62, in bbox_to_xywh_cls_conf
if any(bbox[:, 4] > opt.vis_thresh):
TypeError: list indices must be integers or slices, not tuple

How to deal with this issue? thanks

It seems that the Kalman Filter state for all the tracklets is predicted only when any detections are present in the scene as defined here:

i.e., kf.predict() is called from track.predict() -> tracker.predict() that is called from deep_sort.update() which is only called upon presence of detections.

Is this intended behavior? It seems unreasonable, especially for situations where there are long occlusions, so you would want your KF state to be updated on every frame.

How to get track_id in MOT17-{}.txt? because it's awlays ‘-1’ in the demo。thanks in advance!

how to use the webcam?
det.open('webcam') is wrong

Thank u for your excellent work! I encounter a problem as shown below.

<module 'sys' (built-in)>
Fix size testing.
training chunk_sizes: [32]
The output will be saved to  /home/user-zhm/centerNet-deep-sort-master/CenterNet/src/lib/../../exp/ctdet/default
heads {'hm': 80, 'wh': 2, 'reg': 2}
/home/user-zhm/anaconda3/envs/CenterNet/lib/python3.6/site-packages/sklearn/utils/linear_assignment_.py:21: DeprecationWarning: The linear_assignment_ module is deprecated in 0.21 and will be removed from 0.23. Use scipy.optimize.linear_sum_assignment instead.
  DeprecationWarning)
Creating model...
loaded ./CenterNet/models/ctdet_coco_dla_2x.pth, epoch 230
Drop parameter base.fc.weight.
Drop parameter base.fc.bias.
Loading weights from deep/checkpoint/ckpt.t7... Done!
THCudaCheck FAIL file=/opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument
centernet time: 0.7754421234130859s, fps: 1.2895868947620863, avg fps : 1.2895868947620863
Traceback (most recent call last):
  File "demo_centernet_deepsort.py", line 169, in <module>
    det.detect()
  File "demo_centernet_deepsort.py", line 127, in detect
    results = self.detector.run(im)['results']
  File "/home/user-zhm/centerNet-deep-sort-master/CenterNet/src/lib/detectors/base_detector.py", line 116, in run
    output, dets, forward_time = self.process(images, return_time=True)
  File "/home/user-zhm/centerNet-deep-sort-master/CenterNet/src/lib/detectors/ctdet.py", line 26, in process
    output = self.model(images)[-1]
  File "/home/user-zhm/anaconda3/envs/CenterNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user-zhm/centerNet-deep-sort-master/CenterNet/src/lib/models/networks/pose_dla_dcn.py", line 481, in forward
    z[head] = self.__getattr__(head)(y[-1])
  File "/home/user-zhm/anaconda3/envs/CenterNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user-zhm/anaconda3/envs/CenterNet/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
    input = module(input)
  File "/home/user-zhm/anaconda3/envs/CenterNet/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user-zhm/anaconda3/envs/CenterNet/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 301, in forward
    self.padding, self.dilation, self.groups)
RuntimeError: CUDA error: out of memory

The error is out of memory but I use nvidia-smi to check the memory and find the memory is enough.

GPU: gtx 2080Ti 11G

<module 'sys' (built-in)>
Traceback (most recent call last):
File "D:/Projects/Track/OpenCV/centerNet-deep-sort/demo_centernet_deepsort.py", line 15, in
from CenterNet.src.lib.detectors.detector_factory import detector_factory
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\detectors\detector_factory.py", line 5, in
from .exdet import ExdetDetector
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\detectors\exdet.py", line 33, in
from .base_detector import BaseDetector
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\detectors\base_detector.py", line 14, in
from ..models.model import create_model, load_model
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\models\model.py", line 12, in
from .networks.pose_dla_dcn import get_pose_net as get_dla_dcn
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\models\networks\pose_dla_dcn.py", line 16, in
from .DCNv2.dcn_v2 import DCN
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\models\networks\DCNv2\dcn_v2.py", line 11, in
from .dcn_v2_func import DCNv2Function
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\models\networks\DCNv2\dcn_v2_func.py", line 9, in
from ._ext import dcn_v2 as backend
File "D:\Projects\Track\OpenCV\centerNet-deep-sort\CenterNet\src\lib\models\networks\DCNv2_ext\dcn_v2_init.py", line 6, in
from ..dcn_v2 import lib as _lib, ffi as _ffi
ImportError: cannot import name 'lib'

How did you do to combine the Centernet and deepsort to achieve object tracking ? Can you tell me the exactly procedure? How should i do if i want to combine RefineDet and deepsort to achieve object tracking ? Thanks

Can it use libtorch for forward reasoning?
thanks

The reason of this request is that following error:

"raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead."

Also, some other detectors such as Mask-RCNN is also supported by PyToch1.0.

Thanks

python demo_centernet_deepsort.py

then i got these ~

<module 'sys' (built-in)>
Traceback (most recent call last):
File "demo_centernet_deepsort.py", line 10, in
from detectors.detector_factory import detector_factory
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/detectors/detector_factory.py", line 5, in
from .exdet import ExdetDetector
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/detectors/exdet.py", line 26, in
from .base_detector import BaseDetector
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/detectors/base_detector.py", line 11, in
from models.model import create_model, load_model
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/models/model.py", line 12, in
from .networks.pose_dla_dcn import get_pose_net as get_dla_dcn
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/models/networks/pose_dla_dcn.py", line 16, in
from .DCNv2.dcn_v2 import DCN
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/models/networks/DCNv2/dcn_v2.py", line 11, in
from .dcn_v2_func import DCNv2Function
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/models/networks/DCNv2/dcn_v2_func.py", line 9, in
from ._ext import dcn_v2 as _backend
File "/home/joe/Desktop/centerNet-deep-sort/CenterNet/src/lib/models/networks/DCNv2/_ext/dcn_v2/init.py", line 2, in
from torch.utils.ffi import _wrap_function
File "/home/joe/anaconda3/envs/CenterNet_test/lib/python3.6/site-packages/torch/utils/ffi/init.py", line 1, in
raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.

Could not find a version that satisfies the requirement mkl-fft==1.0.12

==================================== ERRORS ====================================
__________________________ ERROR collecting train.py ___________________________
train.py:44: in
torchvision.datasets.ImageFolder(train_dir, transform=transform_train),
../../anaconda3/envs/CenterNet/lib/python3.6/site-packages/torchvision/datasets/folder.py:178: in init
target_transform=target_transform)
../../anaconda3/envs/CenterNet/lib/python3.6/site-packages/torchvision/datasets/folder.py:79: in init
"Supported extensions are: " + ",".join(extensions)))
E RuntimeError: Found 0 files in subfolders of: /home/ying/centerNet-deep-sort-master/deep/data/train
E Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.51 seconds ============================
Process finished with exit code 0

thanks for your contribution, Pytorch has updated to 1.3 now . So I try to run your code in pytorch1.1.0, but I went some bugs .
the main bug is in CenterNet/src/lib/models/networks/DCNv2/_ext/dcn_v2
/init.py:
error:ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.

I tried to fix it according to "pytorch/pytorch#15645" but failed , So you have some advice or Do you have plan to rewritte it in pytorch>=1.0 ?

Tried to execute

python deep/train.py .

Code got crashed at https://github.com/kimyoon-young/centerNet-deep-sort/blob/master/deep/model.py#L82
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=405 error=11 : invalid argument
Traceback (most recent call last):
File "/snap/pycharm-community/179/plugins/python-ce/helpers/pydev/pydevd.py", line 1434, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/snap/pycharm-community/179/plugins/python-ce/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/sdk/Analysis/centerNet-deep-sort-master/deep/train.py", line 194, in
main()
File "/home/sdk/Analysis/centerNet-deep-sort-master/deep/train.py", line 186, in main
train_loss, train_err = train(epoch)
File "/home/sdk/Analysis/centerNet-deep-sort-master/deep/train.py", line 89, in train
outputs = net(inputs)
File "/home/sdk/anaconda3/envs/reinforce_learn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "./model.py", line 84, in forward
x = self.conv(x)
File "/home/sdk/anaconda3/envs/reinforce_learn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/sdk/anaconda3/envs/reinforce_learn/lib/python3.6/site-packages/torch/nn/modules/container.py", line 92, in forward
input = module(input)
File "/home/sdk/anaconda3/envs/reinforce_learn/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/sdk/anaconda3/envs/reinforce_learn/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 320, in forward
self.padding, self.dilation, self.groups)
RuntimeError: cuda runtime error (11) : invalid argument at /pytorch/aten/src/THC/THCGeneral.cpp:405

I am wondering why MOTA and MOTP result of CenterNet-based(DLA34) deepsort are lower than Yolov3-based deepsort's in MOT16?
Can anyone explain about that？ pls

I have a problem with file path for the imports
I did change the CENTERNET_PATH accordingly, like int the instruction
CENTERNET_PATH = '/home/ai/Downloads/centerNet-deep-sort-master/CenterNet/src/lib/'

thanks for answer

Nice repo @kimyoon-young. Very useful. Thank you.

Btw, I see there is yolov3 model in this repo.
How to use yolov3 as the detector? instead of
#ARCH = 'dla_34' and 'resdcn_18'