facebookresearch / detectandtrack Goto Github PK
View Code? Open in Web Editor NEWThe implementation of an algorithm presented in the CVPR18 paper: "Detect-and-Track: Efficient Pose Estimation in Videos"
License: Apache License 2.0
The implementation of an algorithm presented in the CVPR18 paper: "Detect-and-Track: Efficient Pose Estimation in Videos"
License: Apache License 2.0
I0719 20:23:34.495332 4086 net_async_base.cc:435] Using specified CPU pool size: 4; NUMA node id: -1
I0719 20:23:34.495383 4086 net_async_base.cc:440] Created new CPU pool, size: 4; NUMA node id: -1
E0719 20:23:34.497113 4194 net_async_base.cc:368] [enforce fail at conv_op_cudnn.cc:557] filter.dim32(i + 2) == kernel_[i]. 7 vs 1 Error from operator:
input: "gpu_0/data" input: "gpu_0/conv1_w" output: "gpu_0/conv1" name: "" type: "Conv" arg { name: "kernels" ints: 1 ints: 7 ints: 7 } arg { name: "exhaustive_search" i: 0 } arg { name: "order" s: "NCHW" } arg { name: "pads" ints: 0 ints: 3 ints: 3 ints: 0 ints: 3 ints: 3 } arg { name: "strides" ints: 1 ints: 2 ints: 2 } device_option { device_type: 1 cuda_gpu_id: 0 } engine: "CUDNN", op Conv
WARNING workspace.py: 189: Original python traceback for operator 2
in network keypoint_rcnn
in exception above (most recent call last):
WARNING workspace.py: 194: File "tools/test_net.py", line 145, in
WARNING workspace.py: 194: File "tools/test_net.py", line 106, in main
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/core/test_engine.py", line 322, in test_net_on_dataset
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/core/test_engine.py", line 135, in test_net
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/core/test_engine.py", line 60, in initialize_model_from_cfg
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/model_builder.py", line 61, in create
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/model_builder.py", line 154, in keypoint_rcnn
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/model_builder.py", line 305, in build_generic_fast_rcnn_model
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/model_builder.py", line 951, in build_data_parallel_model
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/model_builder.py", line 197, in _single_gpu_build_func
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/FPN3D.py", line 51, in add_fpn_ResNet101_conv5_body
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/FPN3D.py", line 98, in add_fpn_generic_onto_body
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/ResNet3D.py", line 389, in add_ResNet101_conv5_body
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/DetectAndTrack/lib/modeling/ResNet3D.py", line 261, in add_ResNet_convX_body
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/anaconda2/envs/detect_and_track/lib/python2.7/site-packages/caffe2/python/cnn.py", line 86, in ConvNd
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/anaconda2/envs/detect_and_track/lib/python2.7/site-packages/caffe2/python/brew.py", line 107, in scope_wrapper
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/anaconda2/envs/detect_and_track/lib/python2.7/site-packages/caffe2/python/helpers/conv.py", line 164, in conv_nd
WARNING workspace.py: 194: File "/home/Hello/workspace/tracking/anaconda2/envs/detect_and_track/lib/python2.7/site-packages/caffe2/python/helpers/conv.py", line 123, in ConvBase
Traceback (most recent call last):
File "tools/test_net.py", line 145, in
main(ind_range=args.range, multi_gpu_testing=False)
File "tools/test_net.py", line 106, in main
engine.test_net_on_dataset(multi_gpu=multi_gpu_testing)
File "/home/Hello/workspace/tracking/DetectAndTrack/lib/core/test_engine.py", line 322, in test_net_on_dataset
all_boxes, all_segms, all_keyps = test_net()
File "/home/Hello/workspace/tracking/DetectAndTrack/lib/core/test_engine.py", line 160, in test_net
model, im, box_proposals, timers)
File "/home/Hello/workspace/tracking/DetectAndTrack/lib/core/test.py", line 907, in im_detect_all
scores, boxes, im_scales = im_detect_bbox(model, im, box_proposals)
File "/home/Hello/workspace/tracking/DetectAndTrack/lib/core/test.py", line 197, in im_detect_bbox
workspace.RunNet(model.net.Proto().name)
File "/home/Hello/workspace/tracking/anaconda2/envs/detect_and_track/lib/python2.7/site-packages/caffe2/python/workspace.py", line 221, in RunNet
StringifyNetName(name), num_iter, allow_fail,
File "/home/Hello/workspace/tracking/anaconda2/envs/detect_and_track/lib/python2.7/site-packages/caffe2/python/workspace.py", line 181, in CallWithExceptionIntercept
return func(*args, **kwargs)
RuntimeError: [enforce fail at conv_op_cudnn.cc:557] filter.dim32(i + 2) == kernel[i]. 7 vs 1 Error from operator:
input: "gpu_0/data" input: "gpu_0/conv1_w" output: "gpu_0/conv1" name: "" type: "Conv" arg { name: "kernels" ints: 1 ints: 7 ints: 7 } arg { name: "exhaustive_search" i: 0 } arg { name: "order" s: "NCHW" } arg { name: "pads" ints: 0 ints: 3 ints: 3 ints: 0 ints: 3 ints: 3 } arg { name: "strides" ints: 1 ints: 2 ints: 2 } device_option { device_type: 1 cuda_gpu_id: 0 } engine: "CUDNN"
What does this error mean,and how can I resolve this?
hi please change:
git clone https://github.com/rohitgirdhar/DetectAndTrack.git
to
git clone https://github.com/facebookresearch/DetectAndTrack.git
in the installation instructions
the image datasets PoseTrack 2018 and PoseTrack 2017,which one should be used?
I have downloaded the Annotation data “labels_v0.75”,and "test.tar"。so how can the "gen_posetrack_json.py" be used?
pads=2 * [0, 3, 3] = [0, 3, 3, 0, 3, 3], why does it has 6 dimensions, what does "0" mean? what does "1" mean in " strides=[1, 2, 2]"?
Hey,
This is more of a question than an issue. Feel free to delete this if it's not the right place. I want to know, how can I actually detect and track a specific part of the body, say a tshirt and be able to add something on top of it.
So for example, adding a logo (image) on actor's tshirt in such a way where it looks like the actor is actually wearing a shirt with the logo on it (so the logo image actually moves as the actor moves in the video frames).
Is this possible? If not, please point me in the right direction.
Thanks in advance!
Thanks for your answer
hi ~ i have a issue, could you help me ?
i run the commond in the README.md
'''''bash
python launch.py
--cfg configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml
--mode test
TEST.WEIGHTS pretrained models/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/model_final.pkl
'''''''
but it has the log......, i am using linux ubuntu16.04, and have 4 GPU (TITAN XP 11G)
********* it is log **********
tools/test_net.py: 10: tools/test_net.py: Test a Fafst R-CNN network on an image database: not found
from: cannt read /var/mail/future
from: cannt read /var/mail/caffe2.python
tool/test_net.py:21: tools/test_net.py:syntax error;"(" unexpected
Traceback(most recennt call last);
File "tools/test_net.py", line 143, in
main(ind_rang=args.range, multi_gpu_testing=args.multi_gpu_testing)
File "tools/test_net.py", line 104, in main
engine.test_net_on_dataset(multi_gpu=multi_gpu_testing)
File "data/DetectAndTrack/lib/core/test_engine.py", line 319, in test_net_on_dataset
num_images, output_dir)
File "data/DetectAndTrack/lib/core/test_engine.py", line 284, in multi_gpu_test_net_on_dataset
'detection' , num_images, binary, output_dir
File "data/DetectAndTrack/lib/utils/subprocess.py", line 67, in process_in_parallel
log_subprocess_output(i, p, output_dir, tag, start, end)
File "data/DetectAndTrack/lib/utils/subprocess.py", line 97, in log_subprocess_output
assert ret == 0, 'Range subprocess failed (exit code: {})'.format(ret)
AssertionError:Range subprocess failed(exit code : 2)
Thank you for releasing the codes. After installing caffe2 and this repo, I would like to run the testing as below
export CUDA_VISIBLE_DEVICES="0,1,2,3"
python launch.py \
--cfg configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml \
--mode test \
TEST.WEIGHTS /data/action/DetectAndTrack/pretrained_models/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/model_final.pkl
However, I got the following errors.
Found Detectron ops lib: /usr/local/lib/libcaffe2_detectron_ops_gpu.so
E0507 21:06:30.171579 6159 init_intrinsics_check.cc:43] CPU feature avx is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0507 21:06:30.171617 6159 init_intrinsics_check.cc:43] CPU feature avx2 is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
E0507 21:06:30.171631 6159 init_intrinsics_check.cc:43] CPU feature fma is present on your machine, but the Caffe2 binary is not compiled with it. It means you may not get the full speed of your CPU.
INFO test_net.py: 109: Called with args:
INFO test_net.py: 110: Namespace(cfg_file='configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml', multi_gpu_testing=True, opts=['OUTPUT_DIR', 'outputs/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml', 'TEST.WEIGHTS', '/data/action/DetectAndTrack/pretrained_models/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/model_final.pkl'], range=None, vis=False, wait=True)
INFO test_net.py: 116: Testing with config:
INFO test_net.py: 117: {'BBOX_XFORM_CLIP': 4.135166556742356,
'CLUSTER': {'AUTO_RESUME': True, 'ON_CLUSTER': False},
'DEBUG': {'DATA_LOADING': False, 'STOP_TRAIN_ITER': False},
'DEDUP_BOXES': 0.0625,
'DEVSTORAGE': {'HOSTNAME': '',
'MOUNT_ENABLED': False,
'MOUNT_POINT': '/tmp/devstorage',
'REMOTE_PATH': ''},
'EPS': 1e-14,
'EVAL': {'EVAL_MPII_DROP_DETECTION_THRESHOLD': 0.5,
'EVAL_MPII_KPT_THRESHOLD': 1.95,
'EVAL_MPII_PER_VIDEO': False},
'EXT_PATHS': {'POSEVAL_CODE_PATH': '/home/rgirdhar/local/OpenSource/bitbucket/poseval/'},
'FAST_RCNN': {'MLP_HEAD_DIM': 1024,
'ROI_XFORM_METHOD': 'RoIAlign',
'ROI_XFORM_RESOLUTION': 7,
'ROI_XFORM_SAMPLING_RATIO': 2},
'FINAL_MSG': '',
'FPN': {'COARSEST_STRIDE': 32,
'DIM': 256,
'EXTRA_CONV_LEVELS': False,
'FPN_ON': True,
'INPLACE_LATERAL': False,
'MULTILEVEL_ROIS': True,
'MULTILEVEL_RPN': True,
'ROI_CANONICAL_LEVEL': 4,
'ROI_CANONICAL_SCALE': 224,
'ROI_MAX_LEVEL': 5,
'ROI_MIN_LEVEL': 2,
'RPN_ANCHOR_START_SIZE': 32,
'RPN_ASPECT_RATIOS': (0.5, 1, 2),
'RPN_MAX_LEVEL': 6,
'RPN_MIN_LEVEL': 2,
'ZERO_INIT_LATERAL': False},
'KRCNN': {'CONV_HEAD_DIM': 512,
'CONV_HEAD_KERNEL': 3,
'CONV_INIT': 'MSRAFill',
'DECONV_DIM': 256,
'DECONV_KERNEL': 4,
'DILATION': 1,
'HEATMAP_SIZE': 56,
'INFERENCE_MIN_SIZE': 0,
'KEYPOINT_CONFIDENCE': 'bbox',
'LOSS_WEIGHT': 1.0,
'MIN_KEYPOINT_COUNT_FOR_VALID_MINIBATCH': 20,
'NMS_OKS': False,
'NO_3D_DECONV_TIME_TO_CH': False,
'NUM_KEYPOINTS': 17,
'NUM_STACKED_CONVS': 8,
'ROI_KEYPOINTS_HEAD': 'keypoint_rcnn_heads.add_roi_pose_head_v1convX',
'ROI_XFORM_METHOD': 'RoIAlign',
'ROI_XFORM_RESOLUTION': 14,
'ROI_XFORM_SAMPLING_RATIO': 2,
'UP_SCALE': 2,
'USE_3D_DECONV': False,
'USE_DECONV': False,
'USE_DECONV_OUTPUT': True},
'MATLAB': 'matlab',
'MEMONGER': True,
'MEMONGER_SHARE_ACTIVATIONS': False,
'MODEL': {'BBOX_REG_WEIGHTS': (10.0, 10.0, 5.0, 5.0),
'BN_EPSILON': 1.0000001e-05,
'BN_MOMENTUM': 0.9,
'CLS_AGNOSTIC_BBOX_REG': False,
'CONV_BODY': 'FPN3D.add_fpn_ResNet101_conv5_body',
'DILATION': 1,
'EXECUTION_TYPE': 'dag',
'FASTER_RCNN': True,
'KEYPOINTS_ON': True,
'MASK_ON': False,
'NUM_CLASSES': 2,
'PS_GRID_SIZE': 3,
'ROI_HEAD': 'head_builder.add_roi_2mlp_head',
'RPN_ONLY': False,
'TYPE': 'keypoint_rcnn',
'USE_BN': False,
'USE_BN_TESTMODE_ONLY': False,
'VIDEO_ON': True},
'MRCNN': {'CLS_SPECIFIC_MASK': True,
'CONV_INIT': 'GaussianFill',
'DILATION': 2,
'DIM_REDUCED': 256,
'MASK_HEAD_NAME': '',
'RESOLUTION': 14,
'ROI_XFORM_METHOD': 'RoIAlign',
'ROI_XFORM_RESOLUTION': 7,
'ROI_XFORM_SAMPLING_RATIO': 0,
'THRESH_BINARIZE': 0.5,
'UPSAMPLE_RATIO': 1,
'USE_FC_OUTPUT': False,
'WEIGHT_LOSS_MASK': 1.0},
'NUM_GPUS': 4,
'NUM_WORKERS': 4,
'OUTPUT_DIR': 'outputs/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml',
'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]),
'RESNETS': {'NUM_GROUPS': 1,
'STRIDE_1X1': True,
'TRANS_FUNC': 'bottleneck_transformation',
'WIDTH_PER_GROUP': 64},
'RNG_SEED': 3,
'ROIDB_SUBSET': [],
'ROOT_DIR': '/data/action/DetectAndTrack',
'ROOT_GPU_ID': 0,
'RPN': {'ASPECT_RATIOS': (0.5, 1, 2),
'ON': True,
'SIZES': (64, 128, 256, 512),
'STRIDE': 16},
'SOLVER': {'BASE_LR': 0.0005,
'GAMMA': 0.1,
'LOG_LR_CHANGE_THRESHOLD': 1.1,
'LRS': [],
'LR_POLICY': 'steps_with_decay',
'MAX_ITER': 26000,
'MOMENTUM': 0.9,
'SCALE_MOMENTUM': True,
'SCALE_MOMENTUM_THRESHOLD': 1.1,
'STEPS': [0, 20000, 24000],
'STEP_SIZE': 30000,
'WARM_UP_FACTOR': 0.3333333333333333,
'WARM_UP_ITERS': 500,
'WARM_UP_METHOD': u'linear',
'WEIGHT_DECAY': 0.0001},
'TEST': {'BBOX_AUG': {'AREA_TH_HI': 32400,
'AREA_TH_LO': 2500,
'ASPECT_RATIOS': (),
'ASPECT_RATIO_H_FLIP': False,
'COORD_HEUR': 'ID',
'H_FLIP': False,
'MAX_SIZE': 4000,
'SCALES': (),
'SCALE_H_FLIP': False,
'SCALE_SIZE_DEP': False,
'SCORE_HEUR': 'ID'},
'BBOX_REG': True,
'BBOX_VOTE': {'ENABLED': False, 'VOTE_TH': 0.8},
'COMPETITION_MODE': False,
'DATASET': 'posetrack_v1.0_val',
'DATASETS': (),
'DETECTIONS_PER_IM': 100,
'ENSEMBLE': {'DEVSTORAGE_CACHE': False,
'PROPOSAL_CACHE': '/tmp',
'RPN_CONFIGS': ()},
'EXT_CNN_FEATURES': False,
'EXT_CNN_FEATURES_MODEL': 'ImNet',
'FORCE_JSON_DATASET_EVAL': False,
'INIT_RANDOM_VARS_BEFORE_LOADING': False,
'KPS_AUG': {'AREA_TH': 32400,
'ASPECT_RATIOS': (),
'ASPECT_RATIO_H_FLIP': False,
'HEUR': 'HM_AVG',
'H_FLIP': False,
'MAX_SIZE': 4000,
'SCALES': (),
'SCALE_H_FLIP': False,
'SCALE_SIZE_DEP': False},
'MASK_AUG': {'AREA_TH': 32400,
'ASPECT_RATIOS': (),
'ASPECT_RATIO_H_FLIP': False,
'HEUR': 'SOFT_AVG',
'H_FLIP': False,
'MAX_SIZE': 4000,
'SCALES': (),
'SCALE_H_FLIP': False,
'SCALE_SIZE_DEP': False},
'MAX_SIZE': 1333,
'NMS': 0.5,
'PROPOSAL_FILE': '',
'PROPOSAL_FILES': (),
'PROPOSAL_LIMIT': 2000,
'RPN_MIN_SIZE': 0,
'RPN_NMS_THRESH': 0.7,
'RPN_POST_NMS_TOP_N': 1000,
'RPN_PRE_NMS_TOP_N': 1000,
'SCALES': (800,),
'SCORE_THRESH': 0.05,
'SOFT_NMS': {'ENABLED': False, 'METHOD': 'linear', 'SIGMA': 0.5},
'SVM': False,
'WEIGHTS': '/data/action/DetectAndTrack/pretrained_models/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/model_final.pkl'},
'TRACKING': {'BIPARTITE_MATCHING_ALGO': 'hungarian',
'CNN_MATCHING_LAYER': 'layer3',
'CONF_FILTER_INITIAL_DETS': 0.95,
'DEBUG': {'DUMMY_TRACKS': False,
'FLOW_SMOOTHING_COMBINE': False,
'UPPER_BOUND': False,
'UPPER_BOUND_2_GT_KPS': False,
'UPPER_BOUND_2_GT_KPS_ONLY_CONF': False,
'UPPER_BOUND_3_SHOTS': False,
'UPPER_BOUND_4_EVAL_UPPER_BOUND': False,
'UPPER_BOUND_5_GT_KPS_ONLY': False},
'DETECTIONS_FILE': '',
'DISTANCE_METRICS': ('bbox-overlap', 'cnn-cosdist'),
'DISTANCE_METRIC_WTS': (1.0, 0.0),
'FLOW_SMOOTHING': {'EXTEND_TRACKS': True,
'FLOW_SHOT_BOUNDARY_TH': 6.0,
'N_CONTEXT_FRAMES': 3},
'FLOW_SMOOTHING_ON': False,
'KEEP_CENTER_DETS_ONLY': True,
'KP_CONF_TYPE': 'global',
'LSTM': {'BATCH_SIZE': 20,
'CONSIDER_SHORT_TRACKS_TOO': False,
'DROPOUT': 0.2,
'EMSIZE': 200,
'EPOCHS': 10,
'FEATS_TO_CONSIDER': [u'bbox', u'kpts'],
'GRAD_CLIP': 0.25,
'LOG_INTERVAL': 200,
'LOSS_LAST_PRED_ONLY': False,
'LR': 0.1,
'MODEL': 'LSTM',
'NHID': 200,
'NLAYERS': 2,
'NUM_WORKERS': 4,
'TIED_WTS': False},
'LSTM_TEST': {'LSTM_TRACKING_ON': False, 'LSTM_WEIGHTS': ''}},
'TRAIN': {'ASPECT_GROUPING': True,
'BATCH_SIZE_PER_IM': 512,
'BBOX_NORMALIZE_TARGETS_PRECOMPUTED_deprecated': None,
'BBOX_REG': True,
'BBOX_THRESH': 0.5,
'BG_THRESH_HI': 0.5,
'BG_THRESH_LO': 0.0,
'CROWD_FILTER_THRESH': 0.7,
'DATASET': 'posetrack_v1.0_train',
'DROPOUT': 0.0,
'FG_FRACTION': 0.25,
'FG_THRESH': 0.5,
'GT_MIN_AREA': -1,
'IMS_PER_BATCH': 1,
'MAX_SIZE': 1333,
'MINIBATCH_QUEUE_SIZE': 64,
'PROPOSAL_FILE': '',
'RPN_BATCH_SIZE_PER_IM': 256,
'RPN_FG_FRACTION': 0.5,
'RPN_MIN_SIZE': 0,
'RPN_NEGATIVE_OVERLAP': 0.3,
'RPN_NMS_THRESH': 0.7,
'RPN_POSITIVE_OVERLAP': 0.7,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 2000,
'RPN_STRADDLE_THRESH': 0,
'SCALES': (640, 672, 704, 736, 768, 800),
'SNAPSHOT_ITERS': 20000,
'USE_FLIPPED': True,
'WEIGHTS': 'pretrained_models/e2e_keypoint_rcnn_R-101-FPN_1x_trainedCOCO.pkl'},
'USE_GPU_NMS_deprecated': None,
'USE_NCCL': False,
'VIDEO': {'BODY_HEAD_LINK': 'slice-center',
'DEBUG_USE_RPN_GT': False,
'DEFAULT_CLIPS_PER_VIDEO': 9999999999,
'NUM_FRAMES': 1,
'NUM_FRAMES_MID': 1,
'PREDICT_RPN_BOX_VIS': False,
'RPN_TUBE_GEN_STYLE': 'replicate',
'TIME_INTERVAL': 0,
'TIME_KERNEL_DIM': {'BODY': 1,
'HEAD_DET': 1,
'HEAD_KPS': 1,
'HEAD_RPN': 1},
'TIME_STRIDE_ON': False,
'WEIGHTS_INFLATE_MODE': 'center-only'},
'VIS': False,
'VIS_THR': 0.9,
'VOC_DIR': '/mnt/vol/gfsai-east/ai-group/datasets'}
/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "tools/test_net.py", line 142, in <module>
main(ind_range=args.range, multi_gpu_testing=args.multi_gpu_testing)
File "tools/test_net.py", line 69, in main
import core.test_engine as engine
File "/data/action/DetectAndTrack/lib/core/test_engine.py", line 34, in <module>
from modeling import model_builder
File "/data/action/DetectAndTrack/lib/modeling/model_builder.py", line 18, in <module>
from modeling.detector import DetectionModelHelper
File "/data/action/DetectAndTrack/lib/modeling/detector.py", line 21, in <module>
from ops.roi_blob_transforms import RoIToBatchFormatOp
ImportError: No module named roi_blob_transforms
It looks like there is a few missing files such as roi_blob_transforms
and debug_ops
in folder ops
.
Could you please let me know how to solve this problem?
Thanks
Hi,
Is it possible to extract the detected key points and tracking information without evaluating performance of the model? I'd like to use Detect and Track on a data set for which I don't have the ground-truth annotations. Thank you!
[ 62%] Built target caffe2_detectron_custom_ops
-- Removing ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/./caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o
-- Removing ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/./caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o
/usr/local/bin/cmake -E remove ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/./caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o
/usr/local/bin/cmake -E remove ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/./caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o
-- Generating dependency file: ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o.NVCC-depend
-- Generating dependency file: ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o.NVCC-depend
/usr/local/cuda/bin/nvcc -M -D__CUDACC__ ....../DetectAndTrack/lib/ops/zero_even_op.cu -o ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o.NVCC-depend -ccbin /usr/bin/cc -m64 -Dcaffe2_detectron_custom_ops_gpu_EXPORTS -Xcompiler ,\"-fPIC\" -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_52,code=sm_52 -Xcudafe --diag_suppress=cc_clobber_ignored -Xcudafe --diag_suppress=integer_sign_change -Xcudafe --diag_suppress=useless_using_declaration -Xcudafe --diag_suppress=set_but_not_used -std=c++11 -Xcompiler -fPIC --expt-relaxed-constexpr --expt-extended-lambda -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored -Xcudafe --diag_suppress=integer_sign_change -Xcudafe --diag_suppress=useless_using_declaration -Xcudafe --diag_suppress=set_but_not_used -std=c++11 -Xcompiler -fPIC --expt-relaxed-constexpr -DNVCC -I/usr/local/cuda/include -I-I/usr/local/include -I/usr/include/eigen3 -I....../DetectAndTrack/lib/-I/usr/local/include -I/usr/local/include
/usr/local/cuda/bin/nvcc -M -D__CUDACC__ ....../DetectAndTrack/lib/ops/affine_channel_nd_op.cu -o ....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o.NVCC-depend -ccbin /usr/bin/cc -m64 -Dcaffe2_detectron_custom_ops_gpu_EXPORTS -Xcompiler ,\"-fPIC\" -DONNX_NAMESPACE=onnx_c2 -gencode arch=compute_52,code=sm_52 -Xcudafe --diag_suppress=cc_clobber_ignored -Xcudafe --diag_suppress=integer_sign_change -Xcudafe --diag_suppress=useless_using_declaration -Xcudafe --diag_suppress=set_but_not_used -std=c++11 -Xcompiler -fPIC --expt-relaxed-constexpr --expt-extended-lambda -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -Xcudafe --diag_suppress=cc_clobber_ignored -Xcudafe --diag_suppress=integer_sign_change -Xcudafe --diag_suppress=useless_using_declaration -Xcudafe --diag_suppress=set_but_not_used -std=c++11 -Xcompiler -fPIC --expt-relaxed-constexpr -DNVCC -I/usr/local/cuda/include -I-I/usr/local/include -I/usr/include/eigen3 -I....../DetectAndTrack/lib/-I/usr/local/include -I/usr/local/include
nvcc fatal : redefinition of argument 'std'
CMake Error at caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o.Release.cmake:215 (message):
Error generating
....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/./caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o
CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/build.make:73: recipe for target 'CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o' failed
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_zero_even_op.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
nvcc fatal : redefinition of argument 'std'
CMake Error at caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o.Release.cmake:215 (message):
Error generating
....../DetectAndTrack/lib/build/CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/./caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o
CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/build.make:66: recipe for target 'CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o' failed
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/ops/caffe2_detectron_custom_ops_gpu_generated_affine_channel_nd_op.cu.o] Error 1
make[2]: Leaving directory '....../DetectAndTrack/lib/build'
CMakeFiles/Makefile2:112: recipe for target 'CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/all' failed
make[1]: *** [CMakeFiles/caffe2_detectron_custom_ops_gpu.dir/all] Error 2
make[1]: Leaving directory '....../DetectAndTrack/lib/build'
Makefile:132: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
File "tools/test_net.py", line 151, in
main(ind_range=args.range, multi_gpu_testing=False)
File "tools/test_net.py", line 113, in main
engine.test_net_on_dataset(multi_gpu=multi_gpu_testing)
File "/home/chandu/workspace/tracking/DaT/lib/core/test_engine.py", line 321, in test_net_on_dataset
all_boxes, all_segms, all_keyps = test_net()
File "/home/chandu/workspace/tracking/DaT/lib/core/test_engine.py", line 135, in test_net
model = initialize_model_from_cfg()
File "/home/chandu/workspace/tracking/DaT/lib/core/test_engine.py", line 59, in initialize_model_from_cfg
init_params=cfg.TEST.INIT_RANDOM_VARS_BEFORE_LOADING)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/model_builder.py", line 63, in create
return get_func(model_name)(init_model(model_name, train, init_params))
File "/home/chandu/workspace/tracking/DaT/lib/modeling/model_builder.py", line 156, in keypoint_rcnn
add_roi_keypoint_head_func=get_func(cfg.KRCNN.ROI_KEYPOINTS_HEAD))
File "/home/chandu/workspace/tracking/DaT/lib/modeling/model_builder.py", line 307, in build_generic_fast_rcnn_model
build_data_parallel_model(model, _single_gpu_build_func)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/model_builder.py", line 953, in build_data_parallel_model
single_gpu_build_func(model)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/model_builder.py", line 199, in _single_gpu_build_func
blob_conv, dim_conv, spatial_scale_conv = add_conv_body_func(model)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/FPN3D.py", line 51, in add_fpn_ResNet101_conv5_body
ResNet.stage_info_ResNet101_conv5)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/FPN3D.py", line 98, in add_fpn_generic_onto_body
conv_body_func(model)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/ResNet3D.py", line 389, in add_ResNet101_conv5_body
return add_ResNet_convX_body(model, (3, 4, 23, 3), freeze_at=2)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/ResNet3D.py", line 262, in add_ResNet_convX_body
p = model.AffineChannelNd(p, 'res_conv1_bn', dim_out=feat_dims[0], inplace=True)
File "/home/chandu/workspace/tracking/DaT/lib/modeling/detector.py", line 94, in AffineChannelNd
if cfg.MODEL.USE_BN:
File "/home/chandu/workspace/tracking/anaconda2/envs/gtracking/lib/python2.7/site-packages/caffe2/python/core.py", line 2082, in getattr
",".join(workspace.C.nearby_opnames(op_type)) + ']'
AttributeError: Method AffineChannelNd is not a registered operator. Did you mean: [AffineChannel]
Why am I getting this error?
I am too tired to swtich among multiple versions of gcc and cuda. Has anyone tested this code works on gcc5 + cuda8.0?
hi:
i use ubunt16.04, use test_net.py to test the pretrained model.pkl,when run to subprocess.py ,subprocess popen to get the p and when "log_subprocess_output && i ==0",the p.wait lead to a lock,so "assert ret ==0",who can give me a suggestions?
'https://github.com/RLovelett/eigen.git/
does not exists
Thank you.
Hi, how to use the downloaded data to train the model? For training we don't have the "PoseTrackV1.0_Annots_val_json" folder?? Can you share the steps to reproduce the training part?
Undefined names have the possibility of raising NameError at runtime.
flake8 testing of https://github.com/facebookresearch/DetectAndTrack on Python 2.7.14
$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
./lib/core/tracking_engine.py:365:24: F821 undefined name 'lstm_track_utils'
cur_boxposes = lstm_track_utils.encode_box_poses(cur_boxes, cur_poses)
^
./lib/core/tracking_engine.py:369:24: F821 undefined name 'lstm_track_utils'
match_scores = lstm_track_utils.compute_matching_scores(
^
./lib/core/tracking_engine.py:395:9: F821 undefined name 'lstm_track_utils'
lstm_track_utils.update_lstms(
^
./lib/datasets/posetrack/poseval/py/eval_helpers.py:161:18: F821 undefined name 'vals'
cum = getCum(vals)
^
./lib/utils/video_io.py:72:21: F821 undefined name 'read_from_everstore'
blobs = read_from_everstore([self.filename], download=False)
^
./tools/test_net.py:78:13: F821 undefined name 'generate_rpn_on_range'
generate_rpn_on_range(ind_range=ind_range)
^
./tools/test_net.py:90:17: F821 undefined name 'generate_rpn_on_dataset'
generate_rpn_on_dataset(multi_gpu=multi_gpu_testing)
^
7 F821 undefined name 'lstm_track_utils'
7
Hi,
In your yaml files in 2d_best, the conv body of model is "FPN3D.add_fpn_ResNet101_conv5_body". Is there something wrong with it?
Thanks!
Hi! We have some questions while trying to use the model.
We are unable to run the test script.
python launch.py
--cfg configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml
--mode test
TEST.WEIGHTS pretrained_models/configs/video/2d_best/01_R101_best_hungarian.yaml/model_final.pkl
The error is shown below
And how to visualize the results of this model?
when i run the test_on_sigle_video.py in README
it has a issue;
found Detectron ops lib: /usr/local/lib/libcaffe2_detectron_ops_gpu.so
key already registerd
offending key; AffineChannelNd.
sorry ! i don't know why
hello,
To avoid conflicting packages, I can create a new environment in conda, and install all the requirements in there。But can i install caffe2 in the environment like my-ENV_python2???
build on:
Thanks!
I have built caffe2 and detectron on GPU (Ubuntu 16 - AWS EC2) and both are working. I am trying to build DetectandTrack, but I am running into an affine_channel_nd_op.* compile error (see below). Caffe2 build was able to find Eigen (-- Found system Eigen at /usr/include/eigen3). What do you think is the issue here? Path Include?
(dl4cvcaffepy27) ubuntu@ip-172-31-0-76:~$ echo $LD_LIBRARY_PATH
/usr/include/eigen3:/usr/local/lib:/home/ubuntu/torch/install/lib:/usr/local/cuda/lib64:/usr/local/cuda-8.0/lib64
(dl4cvcaffepy27) ubuntu@ip-172-31-0-76:/usr/lib$ find . -name eig*
./python2.7/dist-packages/scipy/sparse/linalg/eigen
./cmake/eigen3
(dl4cvcaffepy27) ubuntu@ip-172-31-0-76:/usr/lib$ cd cmake/eigen3/
(dl4cvcaffepy27) ubuntu@ip-172-31-0-76:/usr/lib/cmake/eigen3$ ll
total 16
drwxr-xr-x 2 root root 4096 Nov 17 04:06 ./
drwxr-xr-x 3 root root 4096 Nov 17 04:06 ../
-rw-r--r-- 1 root root 1251 Jan 10 2016 Eigen3Config.cmake
-rw-r--r-- 1 root root 177 Dec 16 2015 UseEigen3.cmake
(dl4cvcaffepy27) ubuntu@ip-172-31-0-76:/usr/lib/cmake/eigen3$
ubuntu@ip-172-31-0-76:~/DetectAndTrack/lib$ cmake .
-- Caffe2: Cannot find gflags automatically. Using legacy find.
-- Caffe2: Found gflags (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libgflags.so)
-- Caffe2: Cannot find glog automatically. Using legacy find.
-- Caffe2: Found glog (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libglog.so)
-- Caffe2: Found protobuf with new-style protobuf targets.
-- Caffe2: Protobuf version 3.5.0
-- Caffe2: CUDA detected: 8.0
-- Found cuDNN: v7.1.2 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
-- Added CUDA NVCC flags for: sm_20 sm_21 sm_30 sm_35 sm_50 sm_52 sm_60 sm_61
-- CUDA detected: 8.0
-- Added CUDA NVCC flags for: sm_20 sm_21 sm_30 sm_35 sm_50 sm_52 sm_60 sm_61
-- Found libcuda: /usr/local/cuda/lib64/stubs/libcuda.so
-- Found libnvrtc: /usr/local/cuda/lib64/libnvrtc.so
-- Found cuDNN: v7.1.2 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib64/libcudnn.so)
-- Summary:
-- CMake version : 3.5.1
-- CMake command : /usr/bin/cmake
-- System name : Linux
-- C++ compiler : /usr/bin/c++
-- C++ compiler version : 5.4.0
-- CXX flags : -std=c++11 -O2 -fPIC -Wno-narrowing
-- Caffe2 version : 0.8.2
-- Caffe2 include path : /usr/local/include
-- Have CUDA : TRUE
-- CUDA version : 8.0
-- CuDNN version : 7.1.2
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/DetectAndTrack/lib
ubuntu@ip-172-31-0-76:~/DetectAndTrack/lib$ make
[ 12%] Building CXX object CMakeFiles/caffe2_detectron_custom_ops.dir/ops/affine_channel_nd_op.cc.o
In file included from /home/ubuntu/DetectAndTrack/lib/ops/affine_channel_nd_op.h:19:0,
from /home/ubuntu/DetectAndTrack/lib/ops/affine_channel_nd_op.cc:13:
/usr/local/include/caffe2/utils/math.h:19:22: fatal error: Eigen/Core: No such file or directory
compilation terminated.
CMakeFiles/caffe2_detectron_custom_ops.dir/build.make:62: recipe for target 'CMakeFiles/caffe2_detectron_custom_ops.dir/ops/affine_channel_nd_op.cc.o' failed
make[2]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/ops/affine_channel_nd_op.cc.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/caffe2_detectron_custom_ops.dir/all' failed
make[1]: *** [CMakeFiles/caffe2_detectron_custom_ops.dir/all] Error 2
Makefile:127: recipe for target 'all' failed
make: *** [all] Error 2
ubuntu@ip-172-31-0-76:~/DetectAndTrack/lib$
I tried using all_requirements file to create new conda environment but out of them 32 packages are not available for installation. can you provide a working solution for installing this system locally?
thanks in advance.
Caffe2 compilation requires system protobuf
, leveldb
, gflags
, opencv
, etc rather than those installed from conda. Please refer to this issue from Caffe2
.
According to README.md
, we install these packages by conda first, and then compile Caffe2
, which will lead to errors like:
undefined reference to google::protobuf::internal::AssignDescriptors(std::__cxx11::basic_string
undefined reference to leveldb::Status::ToString[abi:cxx11]()
undefined reference to google::protobuf::internal::empty_string_[abi:cxx11]
I've solved this problem by:
Add -DCMAKE_CXX_FLAGS=-D__GLIBCXX_USE_CXX11_ABI=0
to the cmake command.
Use system versions of gflags
, protobuf
, opencv
, or leveldb
(try installing with apt-get
). Uninstall these libraries from Anaconda and try to build again.
Finally successfully build on:
Hello,
How can I train the model using my own dataset?
Please explain how to do that.
thank you
Hi, very nice work! Just discovered that deeply in your code you are refering to this library for pose evaluation https://github.com/cheind/py-motmetrics. Which happens to be a library of mine. It seems like you got it via https://github.com/leonid-pishchulin/poseval/, who refers to it via a submodule. This referall has been lost in your code copy. Would you mind noting a link to the original repository in the Dataset and Evaluation
section?
Hi, when I try to run the same command in #1 , an error occurs:
ImportError: No module named 'core.rpn_generator'
And I cannot find any codes with rpn_generator.
Thanks.
when I run evaluation code, the output results:Multiple Object Tracking (MOT) metrics: [798/1987]
& MOTA & MOTA & MOTA & MOTA & MOTA & MOTA & MOTA & MOTA & MOTP & Prec & Rec \
& Head & Shou & Elb & Wri & Hip & Knee & Ankl & Total& Total& Total& Total\
& 61.7 & 65.4 & 57.1 & 45.6 & 54.3 & nan & nan & nan & 50.1 & nan & nan \
INFO mpii_eval_engine.py: 289: (60.566788034021066, nan)
INFO mpii_eval_engine.py: 290: ...Done in 173.403025866
which the MOTA is NAN. what's the reason ?
I'm hitting an import error when trying to test with:
python launch.py \
--cfg configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml \
--mode test \
TEST.WEIGHTS pretrained_models/configs/video/2d_best/01_R101_best_hungarian.yaml/model_final.pkl
Traceback (most recent call last):
File "tools/test_net.py", line 19, in <module>
from core.rpn_generator import generate_rpn_on_range, generate_rpn_on_dataset
File "/home/thomasbalestri/PlayGround/DetectAndTrack/lib/core/rpn_generator.py", line 21, in <module>
from libfb.py import parutil
ImportError: No module named libfb.py
Is this file missing from the repo?
I've used the code to get the predictions for the test set of PoseTrack2017. After getting 214 predicted json files, I zipped them and uploaded to the offical website. Errors are given as follows
Wrong number of result files! 0 results found in archive, but 214 needed. Please make sure that all files are present in root folder of the zip archive.
Uploaded successfully.
zip archive unpacked successfully!
But I have placed 214 predicted json files of test set at the root folder like
results/
"00001_mpii_new_step1_relpath_5sec_testsub.json"
"00002_mpii_new_step1_relpath_5sec_testsub.json"
***
"**.json"
and zip it in ubuntu using the following command
zip -r results.zip results/
or zip it directly in windows.
I don't know why it fails and wait hopfully for your solution.
I tried to run the command to train the model by myself, then I got errors velow:
loading annotations into memory...
Done (t=1.65s)
creating index...
index created!
INFO roidb.py: 34: Appending horizontally-flipped training examples...
INFO roidb.py: 36: Loaded dataset: posetrack_v1.0_train
INFO roidb.py: 116: Filtered 43940 roidb entries: 59404 -> 15464
Video-fying the roidb: 0%| | 0/15464 [00:00<?, ?it/s]/data/weiming.li/.pyenv/versions/anaconda2-5.2.0/lib/python2.7/site-packages/scipy/sparse/compressed.py:746: SparseEfficiencyWarning: Changing the sparsity structure of a csr_matrix is expensive. lil_matrix is more efficient.
SparseEfficiencyWarning)
Video-fying the roidb: 100%|██████████████████████████| 15464/15464 [03:12<00:00, 80.25it/s]
INFO video.py: 200: Video-fied roidb contains 14458 elements
INFO roidb.py: 56: Computing bounding-box regression targets...
INFO roidb.py: 58: done
INFO roidb.py: 173: Ground-truth class histogram:
INFO roidb.py: 177: 0__background__: 0
INFO roidb.py: 177: 1 person: 99792
INFO roidb.py: 178: --------------
INFO roidb.py: 181: total: 99792
INFO train_net.py: 118: 14458 roidb entries
INFO net.py: 165: Loading from: ./pretrained_models/ResNet18_weights.pkl
INFO net.py: 204: conv1_w loaded from weights file into gpu_0/conv1_w: (64, 3, 7, 7)
WARNING net.py: 150: Workspace blob conv1_w ((64, 3, 1, 7, 7)) loaded with pretrained wts (64, 3, 7, 7) after inflating the weights by center-only mode.
INFO net.py: 204: res_conv1_bn_s loaded from weights file into gpu_0/res_conv1_bn_s: (64,)
INFO net.py: 204: res_conv1_bn_b loaded from weights file into gpu_0/res_conv1_bn_b: (64,)
INFO net.py: 204: res2_0_branch2a_w loaded from weights file into gpu_0/res2_0_branch2a_w: (64, 64, 3, 3)
WARNING net.py: 150: Workspace blob res2_0_branch2a_w ((64, 64, 1, 3, 3)) loaded with pretrained wts (64, 64, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res2_0_branch2a_bn_s loaded from weights file into gpu_0/res2_0_branch2a_bn_s: (64,)
INFO net.py: 204: res2_0_branch2a_bn_b loaded from weights file into gpu_0/res2_0_branch2a_bn_b: (64,)
INFO net.py: 204: res2_0_branch2b_w loaded from weights file into gpu_0/res2_0_branch2b_w: (64, 64, 3, 3)
WARNING net.py: 150: Workspace blob res2_0_branch2b_w ((64, 64, 1, 3, 3)) loaded with pretrained wts (64, 64, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res2_0_branch2b_bn_s loaded from weights file into gpu_0/res2_0_branch2b_bn_s: (64,)
INFO net.py: 204: res2_0_branch2b_bn_b loaded from weights file into gpu_0/res2_0_branch2b_bn_b: (64,)
INFO net.py: 204: res2_1_branch2a_w loaded from weights file into gpu_0/res2_1_branch2a_w: (64, 64, 3, 3)
WARNING net.py: 150: Workspace blob res2_1_branch2a_w ((64, 64, 1, 3, 3)) loaded with pretrained wts (64, 64, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res2_1_branch2a_bn_s loaded from weights file into gpu_0/res2_1_branch2a_bn_s: (64,)
INFO net.py: 204: res2_1_branch2a_bn_b loaded from weights file into gpu_0/res2_1_branch2a_bn_b: (64,)
INFO net.py: 204: res2_1_branch2b_w loaded from weights file into gpu_0/res2_1_branch2b_w: (64, 64, 3, 3)
WARNING net.py: 150: Workspace blob res2_1_branch2b_w ((64, 64, 1, 3, 3)) loaded with pretrained wts (64, 64, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res2_1_branch2b_bn_s loaded from weights file into gpu_0/res2_1_branch2b_bn_s: (64,)
INFO net.py: 204: res2_1_branch2b_bn_b loaded from weights file into gpu_0/res2_1_branch2b_bn_b: (64,)
INFO net.py: 204: res3_0_branch2a_w loaded from weights file into gpu_0/res3_0_branch2a_w: (128, 64, 3, 3)
WARNING net.py: 150: Workspace blob res3_0_branch2a_w ((128, 64, 3, 3, 3)) loaded with pretrained wts (128, 64, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res3_0_branch2a_bn_s loaded from weights file into gpu_0/res3_0_branch2a_bn_s: (128,)
INFO net.py: 204: res3_0_branch2a_bn_b loaded from weights file into gpu_0/res3_0_branch2a_bn_b: (128,)
INFO net.py: 204: res3_0_branch2b_w loaded from weights file into gpu_0/res3_0_branch2b_w: (128, 128, 3, 3)
WARNING net.py: 150: Workspace blob res3_0_branch2b_w ((128, 128, 3, 3, 3)) loaded with pretrained wts (128, 128, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res3_0_branch2b_bn_s loaded from weights file into gpu_0/res3_0_branch2b_bn_s: (128,)
INFO net.py: 204: res3_0_branch2b_bn_b loaded from weights file into gpu_0/res3_0_branch2b_bn_b: (128,)
INFO net.py: 204: res3_0_branch1_w loaded from weights file into gpu_0/res3_0_branch1_w: (128, 64, 1, 1)
WARNING net.py: 150: Workspace blob res3_0_branch1_w ((128, 64, 1, 1, 1)) loaded with pretrained wts (128, 64, 1, 1) after inflating the weights by center-only mode.
INFO net.py: 204: res3_0_branch1_bn_s loaded from weights file into gpu_0/res3_0_branch1_bn_s: (128,)
INFO net.py: 204: res3_0_branch1_bn_b loaded from weights file into gpu_0/res3_0_branch1_bn_b: (128,)
INFO net.py: 204: res3_1_branch2a_w loaded from weights file into gpu_0/res3_1_branch2a_w: (128, 128, 3, 3)
WARNING net.py: 150: Workspace blob res3_1_branch2a_w ((128, 128, 3, 3, 3)) loaded with pretrained wts (128, 128, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res3_1_branch2a_bn_s loaded from weights file into gpu_0/res3_1_branch2a_bn_s: (128,)
INFO net.py: 204: res3_1_branch2a_bn_b loaded from weights file into gpu_0/res3_1_branch2a_bn_b: (128,)
INFO net.py: 204: res3_1_branch2b_w loaded from weights file into gpu_0/res3_1_branch2b_w: (128, 128, 3, 3)
WARNING net.py: 150: Workspace blob res3_1_branch2b_w ((128, 128, 3, 3, 3)) loaded with pretrained wts (128, 128, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res3_1_branch2b_bn_s loaded from weights file into gpu_0/res3_1_branch2b_bn_s: (128,)
INFO net.py: 204: res3_1_branch2b_bn_b loaded from weights file into gpu_0/res3_1_branch2b_bn_b: (128,)
INFO net.py: 204: res4_0_branch2a_w loaded from weights file into gpu_0/res4_0_branch2a_w: (256, 128, 3, 3)
WARNING net.py: 150: Workspace blob res4_0_branch2a_w ((256, 128, 3, 3, 3)) loaded with pretrained wts (256, 128, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res4_0_branch2a_bn_s loaded from weights file into gpu_0/res4_0_branch2a_bn_s: (256,)
INFO net.py: 204: res4_0_branch2a_bn_b loaded from weights file into gpu_0/res4_0_branch2a_bn_b: (256,)
INFO net.py: 204: res4_0_branch2b_w loaded from weights file into gpu_0/res4_0_branch2b_w: (256, 256, 3, 3)
WARNING net.py: 150: Workspace blob res4_0_branch2b_w ((256, 256, 3, 3, 3)) loaded with pretrained wts (256, 256, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res4_0_branch2b_bn_s loaded from weights file into gpu_0/res4_0_branch2b_bn_s: (256,)
INFO net.py: 204: res4_0_branch2b_bn_b loaded from weights file into gpu_0/res4_0_branch2b_bn_b: (256,)
INFO net.py: 204: res4_0_branch1_w loaded from weights file into gpu_0/res4_0_branch1_w: (256, 128, 1, 1)
WARNING net.py: 150: Workspace blob res4_0_branch1_w ((256, 128, 1, 1, 1)) loaded with pretrained wts (256, 128, 1, 1) after inflating the weights by center-only mode.
INFO net.py: 204: res4_0_branch1_bn_s loaded from weights file into gpu_0/res4_0_branch1_bn_s: (256,)
INFO net.py: 204: res4_0_branch1_bn_b loaded from weights file into gpu_0/res4_0_branch1_bn_b: (256,)
INFO net.py: 204: res4_1_branch2a_w loaded from weights file into gpu_0/res4_1_branch2a_w: (256, 256, 3, 3)
WARNING net.py: 150: Workspace blob res4_1_branch2a_w ((256, 256, 3, 3, 3)) loaded with pretrained wts (256, 256, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res4_1_branch2a_bn_s loaded from weights file into gpu_0/res4_1_branch2a_bn_s: (256,)
INFO net.py: 204: res4_1_branch2a_bn_b loaded from weights file into gpu_0/res4_1_branch2a_bn_b: (256,)
INFO net.py: 204: res4_1_branch2b_w loaded from weights file into gpu_0/res4_1_branch2b_w: (256, 256, 3, 3)
WARNING net.py: 150: Workspace blob res4_1_branch2b_w ((256, 256, 3, 3, 3)) loaded with pretrained wts (256, 256, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res4_1_branch2b_bn_s loaded from weights file into gpu_0/res4_1_branch2b_bn_s: (256,)
INFO net.py: 204: res4_1_branch2b_bn_b loaded from weights file into gpu_0/res4_1_branch2b_bn_b: (256,)
INFO net.py: 196: conv_rpn_w not found
INFO net.py: 196: conv_rpn_b not found
INFO net.py: 196: rpn_cls_logits_1_w not found
INFO net.py: 196: rpn_cls_logits_1_b not found
INFO net.py: 196: rpn_bbox_pred_1_w not found
INFO net.py: 196: rpn_bbox_pred_1_b not found
INFO net.py: 204: res5_0_branch2a_w loaded from weights file into gpu_0/res5_0_branch2a_w: (512, 256, 3, 3)
WARNING net.py: 150: Workspace blob res5_0_branch2a_w ((512, 256, 1, 3, 3)) loaded with pretrained wts (512, 256, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res5_0_branch2a_bn_s loaded from weights file into gpu_0/res5_0_branch2a_bn_s: (512,)
INFO net.py: 204: res5_0_branch2a_bn_b loaded from weights file into gpu_0/res5_0_branch2a_bn_b: (512,)
INFO net.py: 204: res5_0_branch2b_w loaded from weights file into gpu_0/res5_0_branch2b_w: (512, 512, 3, 3)
WARNING net.py: 150: Workspace blob res5_0_branch2b_w ((512, 512, 1, 3, 3)) loaded with pretrained wts (512, 512, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res5_0_branch2b_bn_s loaded from weights file into gpu_0/res5_0_branch2b_bn_s: (512,)
INFO net.py: 204: res5_0_branch2b_bn_b loaded from weights file into gpu_0/res5_0_branch2b_bn_b: (512,)
INFO net.py: 204: res5_0_branch1_w loaded from weights file into gpu_0/res5_0_branch1_w: (512, 256, 1, 1)
WARNING net.py: 150: Workspace blob res5_0_branch1_w ((512, 256, 1, 1, 1)) loaded with pretrained wts (512, 256, 1, 1) after inflating the weights by center-only mode.
INFO net.py: 204: res5_0_branch1_bn_s loaded from weights file into gpu_0/res5_0_branch1_bn_s: (512,)
INFO net.py: 204: res5_0_branch1_bn_b loaded from weights file into gpu_0/res5_0_branch1_bn_b: (512,)
INFO net.py: 204: res5_1_branch2a_w loaded from weights file into gpu_0/res5_1_branch2a_w: (512, 512, 3, 3)
WARNING net.py: 150: Workspace blob res5_1_branch2a_w ((512, 512, 1, 3, 3)) loaded with pretrained wts (512, 512, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res5_1_branch2a_bn_s loaded from weights file into gpu_0/res5_1_branch2a_bn_s: (512,)
INFO net.py: 204: res5_1_branch2a_bn_b loaded from weights file into gpu_0/res5_1_branch2a_bn_b: (512,)
INFO net.py: 204: res5_1_branch2b_w loaded from weights file into gpu_0/res5_1_branch2b_w: (512, 512, 3, 3)
WARNING net.py: 150: Workspace blob res5_1_branch2b_w ((512, 512, 1, 3, 3)) loaded with pretrained wts (512, 512, 3, 3) after inflating the weights by center-only mode.
INFO net.py: 204: res5_1_branch2b_bn_s loaded from weights file into gpu_0/res5_1_branch2b_bn_s: (512,)
INFO net.py: 204: res5_1_branch2b_bn_b loaded from weights file into gpu_0/res5_1_branch2b_bn_b: (512,)
INFO net.py: 196: cls_score_1_w not found
INFO net.py: 196: cls_score_1_b not found
INFO net.py: 196: bbox_pred_1_w not found
INFO net.py: 196: bbox_pred_1_b not found
INFO net.py: 196: conv_fcn1_w not found
INFO net.py: 196: conv_fcn1_b not found
INFO net.py: 196: conv_fcn2_w not found
INFO net.py: 196: conv_fcn2_b not found
INFO net.py: 196: conv_fcn3_w not found
INFO net.py: 196: conv_fcn3_b not found
INFO net.py: 196: conv_fcn4_w not found
INFO net.py: 196: conv_fcn4_b not found
INFO net.py: 196: conv_fcn5_w not found
INFO net.py: 196: conv_fcn5_b not found
INFO net.py: 196: conv_fcn6_w not found
INFO net.py: 196: conv_fcn6_b not found
INFO net.py: 196: conv_fcn7_w not found
INFO net.py: 196: conv_fcn7_b not found
INFO net.py: 196: conv_fcn8_w not found
INFO net.py: 196: conv_fcn8_b not found
INFO net.py: 196: kps_score_lowres_w not found
INFO net.py: 196: kps_score_lowres_b not found
INFO net.py: 196: kps_score_prefinal_w not found
INFO net.py: 196: kps_score_prefinal_b not found
INFO net.py: 249: pred_b preserved in workspace (unused)
INFO net.py: 249: pred_w preserved in workspace (unused)
I0814 16:49:46.008280 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.014658 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.020335 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.026283 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.032645 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.038347 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.043714 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.049134 44020 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0814 16:49:46.116132 44020 net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 0.00463044 secs
INFO train_net.py: 144: Outputs saved to: /data/weiming.li/shiqi.dong/DetectAndTrack/outputs/configs/video/3d/04_R-18-3D_PTFromImNet.yaml/train/posetrack_v1.0_train/keypoint_rcnn
Traceback (most recent call last):
File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/utils/coordinator.py", line 41, in stop_on_exception
yield
File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/roi_data/loader.py", line 209, in minibatch_loader2
shared_readonly_dict, lock, mp_cur, mp_perm)
File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/roi_data/loader.py", line 161, in _get_next_minibatch2
blobs, valid = get_minibatch(minibatch_db)
File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/roi_data/minibatch.py", line 56, in get_minibatch
im_blob, im_scales = _get_image_blob(roidb)
File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/roi_data/minibatch.py", line 84, in _get_image_blob
im, cfg.PIXEL_MEANS, [target_size], cfg.TRAIN.MAX_SIZE)
File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/utils/blob.py", line 73, in prep_im_for_blob
im = im.astype(np.float32, copy=False)
AttributeError: 'NoneType' object has no attribute 'astype'
INFO loader.py: 220: Stopping mini-batch loading thread
INFO loader.py: 220: Stopping mini-batch loading thread
INFO loader.py: 220: Stopping mini-batch loading thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 256: Stopping enqueue thread
INFO loader.py: 220: Stopping mini-batch loading thread
INFO loader.py: 309: Pre-filling mini-batch queue...
INFO loader.py: 313: [0/64]
INFO loader.py: 321: Join-ing all worker threads...
INFO loader.py: 323: Join-ing <Process(Process-2, stopped)>
INFO loader.py: 323: Join-ing <Process(Process-3, stopped)>
INFO loader.py: 323: Join-ing <Process(Process-4, stopped)>
INFO loader.py: 323: Join-ing <Process(Process-5, stopped)>
INFO loader.py: 323: Join-ing <Thread(Thread-2, stopped 139693153580800)>
INFO loader.py: 323: Join-ing <Thread(Thread-3, stopped 139693038737152)>
INFO loader.py: 323: Join-ing <Thread(Thread-4, stopped 139693153580800)>
INFO loader.py: 323: Join-ing <Thread(Thread-5, stopped 139693038737152)>
INFO loader.py: 323: Join-ing <Thread(Thread-6, stopped 139693030344448)>
INFO loader.py: 323: Join-ing <Thread(Thread-7, stopped 139693153580800)>
INFO loader.py: 323: Join-ing <Thread(Thread-8, stopped 139693021951744)>
INFO loader.py: 323: Join-ing <Thread(Thread-9, stopped 139693021951744)>
/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/detector.py:598: RuntimeWarning: divide by zero encountered in float_scalars
ratio = np.max((new_lr / cur_lr, cur_lr / new_lr))
INFO detector.py: 602: Changing learning rate 0.000000 -> 0.000333 at iter 0
/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/detector.py:613: RuntimeWarning: divide by zero encountered in float_scalars
ratio = np.max((new_lr / cur_lr, cur_lr / new_lr))
I0814 16:49:48.402222 44020 net_async_base.cc:435] Using specified CPU pool size: 32; NUMA node id: -1
I0814 16:49:48.402253 44020 net_async_base.cc:440] Created new CPU pool, size: 32; NUMA node id: -1
E0814 16:49:48.404006 44960 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
E0814 16:49:48.404012 44958 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
E0814 16:49:48.404014 44963 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
E0814 16:49:48.404011 44962 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
E0814 16:49:48.404040 44959 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
E0814 16:49:48.404011 44964 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
E0814 16:49:48.404023 44965 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
E0814 16:49:48.404013 44961 net_async_base.cc:352] Failed to execute an op: DequeueBlobs
WARNING workspace.py: 187: Original python traceback for operator `2` in network `keypoint_rcnn` in exception above (most recent call last):
WARNING workspace.py: 192: File "tools/train_net.py", line 257, in <module>
WARNING workspace.py: 192: File "tools/train_net.py", line 130, in net_trainer
WARNING workspace.py: 192: File "tools/train_net.py", line 107, in create_model
WARNING workspace.py: 192: File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/model_builder.py", line 61, in create
WARNING workspace.py: 192: File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/model_builder.py", line 154, in keypoint_rcnn
WARNING workspace.py: 192: File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/model_builder.py", line 305, in build_generic_fast_rcnn_model
WARNING workspace.py: 192: File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/model_builder.py", line 916, in build_data_parallel_model
WARNING workspace.py: 192: File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/model_builder.py", line 197, in _single_gpu_build_func
WARNING workspace.py: 192: File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/ResNet3D.py", line 337, in add_ResNet18_conv4_body
WARNING workspace.py: 192: File "/data/weiming.li/shiqi.dong/DetectAndTrack/lib/modeling/ResNet3D.py", line 261, in add_ResNet_convX_body
WARNING workspace.py: 192: File "/data/weiming.li/.pyenv/versions/anaconda2-5.2.0/lib/python2.7/site-packages/caffe2/python/cnn.py", line 86, in ConvNd
WARNING workspace.py: 192: File "/data/weiming.li/.pyenv/versions/anaconda2-5.2.0/lib/python2.7/site-packages/caffe2/python/brew.py", line 107, in scope_wrapper
WARNING workspace.py: 192: File "/data/weiming.li/.pyenv/versions/anaconda2-5.2.0/lib/python2.7/site-packages/caffe2/python/helpers/conv.py", line 164, in conv_nd
WARNING workspace.py: 192: File "/data/weiming.li/.pyenv/versions/anaconda2-5.2.0/lib/python2.7/site-packages/caffe2/python/helpers/conv.py", line 123, in _ConvBase
Traceback (most recent call last):
File "tools/train_net.py", line 257, in <module>
checkpoints = net_trainer()
File "tools/train_net.py", line 174, in net_trainer
workspace.RunNet(model.net.Proto().name)
File "/data/weiming.li/.pyenv/versions/anaconda2-5.2.0/lib/python2.7/site-packages/caffe2/python/workspace.py", line 219, in RunNet
StringifyNetName(name), num_iter, allow_fail,
File "/data/weiming.li/.pyenv/versions/anaconda2-5.2.0/lib/python2.7/site-packages/caffe2/python/workspace.py", line 180, in CallWithExceptionIntercept
return func(*args, **kwargs)
RuntimeError: [enforce fail at pybind_state.cc:1096] success. Error running net keypoint_rcnn
How can I fix it? I hope anyone's help, thanks very much. @rohitgirdhar
I tested the test_on_single_video.py with 3D pretrained model by running following command:
python tools/test_on_single_video.py --cfg configs/video/3d/03_R-18-3D_PTFromCOCO.yaml --video ~/shiqi.dong/try/ava/P90hF2S1JzA.avi --output ./visualization/P90/ TEST.WEIGHTS pretrained_models/configs/video/3d/03_R-18-3D_PTFromCOCO.yaml_dir/model_final.pkl
But got following error:
......
INFO net.py: 249: fc7_b preserved in workspace (unused)
I0807 10:22:48.503657 9865 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0807 10:22:48.505997 9865 net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 5.64e-05 secs
I0807 10:22:48.506840 9865 operator.cc:169] Engine CUDNN is not available for operator MaxPool.
I0807 10:22:48.507918 9865 net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 2.4698e-05 secs
I0807 10:22:48.509433 9865 net_dag_utils.cc:102] Operator graph pruning prior to chain compute took: 1.8558e-05 secs
Processing Detection for Frame 1
Traceback (most recent call last):
File "tools/test_on_single_video.py", line 340, in <module>
main(name_scope, gpu_dev, num_images, args)
File "tools/test_on_single_video.py", line 264, in main
model, im_, None) #TODO: Parallelize detection
File "/data/zairan.wang/shiqi.dong/DetectAndTrack/lib/core/test.py", line 905, in im_detect_all
scores, boxes, im_scales = im_detect_bbox(model, im, box_proposals)
File "/data/zairan.wang/shiqi.dong/DetectAndTrack/lib/core/test.py", line 174, in im_detect_bbox
inputs, im_scales = _get_blobs(im, boxes)
File "/data/zairan.wang/shiqi.dong/DetectAndTrack/lib/core/test.py", line 147, in _get_blobs
blobs['data'], im_scale_factors = _get_image_blob(im)
File "/data/zairan.wang/shiqi.dong/DetectAndTrack/lib/core/test.py", line 73, in _get_image_blob
blob = blob_utils.im_list_to_blob(processed_ims)
File "/data/zairan.wang/shiqi.dong/DetectAndTrack/lib/utils/blob.py", line 66, in im_list_to_blob
blob = image_utils.move_batch_to_time(blob, cfg.VIDEO.NUM_FRAMES)
File "/data/zairan.wang/shiqi.dong/DetectAndTrack/lib/utils/image.py", line 85, in move_batch_to_time
assert(blob.shape[0] % T == 0)
AssertionError
So can we run test_on_single_video.py with 3D pretrained model?@yannadani
Could you update the installation using the new caffe2 and detectron?
First, thank you for the sharing.
Everything went well when I tested PoseTrackV1.0_Annots_val.
But when I tested the test data without detections' annotations, I got some errors:
"ERROR test_engine.py: 333: Evaluation crashed with exception No evaluator for dataset: "
I know this one is because of the lack of annotations.
After that, I got a detection.pkl file
But I can't use the detection file for tracking.
When I run the tracking mode, I got the following error:
"zero-size array to reduction operation minimum which has no identity."
--------the total error messages------
File "tools/compute_tracks.py", line 56, in
main()
File "tools/compute_tracks.py", line 52, in main
run_posetrack_tracking(test_output_dir, json_data)
File "/home/X/Documents/DetectAndTrack/lib/core/tracking_engine.py", line 791, in run_posetrack_tracking
dets_withTracks = compute_matches_tracks(json_data, dets, lstm_model)
File "/home/X/Documents/DetectAndTrack/lib/core/tracking_engine.py", line 707, in compute_matches_tracks
_summarize_track_stats(all_tracks, json_data)
File "/home/X/Documents/DetectAndTrack/lib/core/tracking_engine.py", line 664, in _summarize_track_stats
np.min(all_lengths),
File "/usr/local/lib/python2.7/dist-packages/numpy/core/fromnumeric.py", line 2420, in amin
out=out, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/numpy/core/_methods.py", line 29, in _amin
return umr_minimum(a, axis, None, out, keepdims)
ValueError: zero-size array to reduction operation minimum which has no identity
Hi @rohitgirdhar, I'm now trying to run the training code.
However, when I run this:
python launch.py -c configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml
The error is like:
INFO train_net.py: 144: Outputs saved to: /mnt/data/xiangyiy/Repository/DetectAndTrack/outputs/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/train/posetrack_v1.0_train/keypoint_rcnn
Traceback (most recent call last):
File "/mnt/data/xiangyiy/Repository/DetectAndTrack/lib/utils/coordinator.py", line 41, in stop_on_exception
yield
File "/mnt/data/xiangyiy/Repository/DetectAndTrack/lib/roi_data/loader.py", line 209, in minibatch_loader2
shared_readonly_dict, lock, mp_cur, mp_perm)
File "/mnt/data/xiangyiy/Repository/DetectAndTrack/lib/roi_data/loader.py", line 161, in _get_next_minibatch2
blobs, valid = get_minibatch(minibatch_db)
File "/mnt/data/xiangyiy/Repository/DetectAndTrack/lib/roi_data/minibatch.py", line 56, in get_minibatch
im_blob, im_scales = _get_image_blob(roidb)
File "/mnt/data/xiangyiy/Repository/DetectAndTrack/lib/roi_data/minibatch.py", line 84, in _get_image_blob
im, cfg.PIXEL_MEANS, [target_size], cfg.TRAIN.MAX_SIZE)
File "/mnt/data/xiangyiy/Repository/DetectAndTrack/lib/utils/blob.py", line 73, in prep_im_for_blob
im = im.astype(np.float32, copy=False)
AttributeError: 'NoneType' object has no attribute 'astype'
Hi @rohitgirdhar, we can now run the code (both training and inference) perfectly on posetrack dataset. Thanks for this repo!
Currently, we would like to train the model on our custom dataset, which has different number of keypoints and photo sizes. We don't accutually want to train it from scratch. Could you please offer us some suggestions? Thanks!
Can we use this tracker for any type of objects or only for human body?? And is this the real time tracker?
conda create -c conda-forge --name $ENV_NAME --file all_pkg_versions.txt python=2.7 anaconda
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
Current channels:
Thank you for releasing this repo. Could you point me to a repo or some code that uses Detect-and-Track for subsequent action recognition?
Thank you,
i have same problem with #35 ,and my json_dataset.py as follow:
##############################################################
##############################################################
from future import absolute_import
from future import division
from future import print_function
from future import unicode_literals
import os
import numpy as np
import scipy.sparse
import cPickle as pickle
import copy
from tqdm import tqdm
import math
import utils.boxes as box_utils
from utils.timer import Timer
from pycocotools.coco import COCO
from pycocotools import mask as COCOmask
from core.config import cfg
from utils.general import static_vars
import logging
logger = logging.getLogger(name)
IM_DIR = 'image_directory'
ANN_FN = 'annotation_file'
SPLIT_INTO_FRAMES = 'split_into_frames'
FRAMES_FROM_VIDEO = 'frames_from_video'
COMPUTED_ANNOTATIONS_INFO = 'computed_annotations_info'
ANN_DN = 'annotation_directory'
DATASETS = {
'posetrack_v1.0_train': {
IM_DIR: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/data/PoseTrack/',
ANN_FN: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/lists/PoseTrack/v1.0/posetrack_train.json',
ANN_DN: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/data/PoseTrackV1.0_Annots_train_json/',
},
'posetrack_v1.0_val': {
IM_DIR: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/data/PoseTrack/',
ANN_FN: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/lists/PoseTrack/v1.0/posetrack_val.json',
ANN_DN: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/data/PoseTrackV1.0_Annots_val_json',
},
'posetrack_v1.0_test': {
IM_DIR: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/data/PoseTrack/',
ANN_FN: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/lists/PoseTrack/v1.0/posetrack_test.json',
ANN_DN: '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/data/PoseTrackV1.0_Annots_test_json',
},
}
class JsonDataset(object):
def init(self, name):
assert name in DATASETS.keys(), 'Unknown dataset name'
logger.debug('Creating: {}'.format(name))
self.name = name
self.image_directory = DATASETS[name][IM_DIR]
self.debug_timer = Timer()
self.COCO = COCO(DATASETS[name][ANN_FN])
self.annotation_directory = DATASETS[name][ANN_DN] if ANN_DN in
DATASETS[name] else ''
# Set up dataset classes
category_ids = self.COCO.getCatIds()
categories = [c['name'] for c in self.COCO.loadCats(category_ids)]
self.category_to_id_map = dict(zip(categories, category_ids))
self.classes = ['background'] + categories
self.num_classes = len(self.classes)
self.json_category_id_to_contiguous_id = {
v: i + 1 for i, v in enumerate(self.COCO.getCatIds())}
self.contiguous_category_id_to_json_id = {
v: k for k, v in self.json_category_id_to_contiguous_id.items()}
self._init_keypoints(name=self.name)
# Added by rgirdhar: Used in tracking to know which is head keypoints,
# when using PCK distance to connect the boxes
self.person_cat_info = self.COCO.loadCats([
self.category_to_id_map['person']])[0]
# Added by rgirdhar: Set true if the frames need to be read out of a
# video file
self.frames_from_video = DATASETS[name][FRAMES_FROM_VIDEO] if
FRAMES_FROM_VIDEO in DATASETS[name] else False
self.annotations_info = DATASETS[name][COMPUTED_ANNOTATIONS_INFO] if
COMPUTED_ANNOTATIONS_INFO in DATASETS[name] else None
if self.annotations_info is not None:
self.annotations_info['clip_length'] = self.annotations_info
'clip_length'
def get_roidb(
self, gt=False, proposal_file=None, min_proposal_size=2,
proposal_limit=-1, crowd_filter_thresh=0):
assert gt is True or crowd_filter_thresh == 0, \
'Crowd filter threshold must be 0 if ground-truth annotations ' \
'are not included.'
image_ids = self.COCO.getImgIds()
image_ids.sort()
roidb = copy.deepcopy(self.COCO.loadImgs(image_ids))
if len(cfg.ROIDB_SUBSET) > 0:
roidb = roidb[cfg.ROIDB_SUBSET[0]: cfg.ROIDB_SUBSET[1]]
logger.warning('Using a roidb subset {}'.format(cfg.ROIDB_SUBSET))
annots = []
if SPLIT_INTO_FRAMES in DATASETS[self.name] and DATASETS[
self.name][SPLIT_INTO_FRAMES]:
roidb, annots = self._split_roidb_frames(roidb)
for entry in roidb:
self._prep_roidb_entry(entry)
if gt:
# Include ground-truth object annotations
self.debug_timer.tic()
for entry_id, entry in enumerate(roidb):
self._add_gt_annotations(entry, entry_id, annots)
logger.debug('_add_gt_annotations took {:.3f}s'.
format(self.debug_timer.toc(average=False)))
if proposal_file is not None:
# Include proposals from a file
self.debug_timer.tic()
self._add_proposals_from_file(
roidb, proposal_file, min_proposal_size, proposal_limit,
crowd_filter_thresh)
logger.debug('_add_proposals_from_file took {:.3f}s'.
format(self.debug_timer.toc(average=False)))
_add_class_assignments(roidb)
return roidb
def _prep_roidb_entry(self, entry):
# Reference back to the parent dataset
entry['dataset'] = self
# Make file_name an abs path
entry['image'] = os.path.join(self.image_directory, entry['file_name'])
entry['flipped'] = False
entry['has_visible_keypoints'] = False
# Empty placeholders
entry['boxes'] = np.empty((0, 4), dtype=np.float32)
entry['tracks'] = np.empty((0, 1), dtype=np.int32)
# head boxes, if available (like in PoseTrack)
entry['head_boxes'] = np.empty((0, 4), dtype=np.float32)
entry['segms'] = []
entry['gt_classes'] = np.empty((0), dtype=np.int32)
entry['seg_areas'] = np.empty((0), dtype=np.float32)
entry['gt_overlaps'] = scipy.sparse.csr_matrix(np.empty(
(0, self.num_classes), dtype=np.float32))
entry['is_crowd'] = np.empty((0), dtype=np.bool)
# 'box_to_gt_ind_map': Shape is (#rois). Maps from each roi to the index
# in the list of rois that satisfy np.where(entry['gt_classes'] > 0)
entry['box_to_gt_ind_map'] = np.empty((0), dtype=np.int32)
if self.keypoints is not None:
entry['gt_keypoints'] = np.empty(
(0, 3, self.num_keypoints), dtype=np.int32)
# Remove unwanted fields if they exist
for k in ['date_captured', 'url', 'license', 'file_name']:
if k in entry:
del entry[k]
def convert_raw_predictions_to_objs(self, annots, image_id):
if len(annots['boxes']) == 0:
return []
objs = []
N = annots['boxes'].shape[0]
for i in range(N):
obj = {}
# COCO labels are in xywh format, but I make predictions in xyxy
# Remove the score from box before converting
obj['bbox'] = box_utils.xyxy_to_xywh(annots['boxes'][i][
np.newaxis, :4]).reshape((-1,)).tolist()
obj['num_keypoints'] = annots['poses'][i].shape[-1]
assert(obj['num_keypoints'] == cfg.KRCNN.NUM_KEYPOINTS)
obj['segmentation'] = []
obj['area'] = obj['bbox'][-1] * obj['bbox'][-2]
obj['iscrowd'] = False
pose = annots['poses'][i][:3].transpose()
pose[pose[:, -1] >= 2.0, -1] = 2
pose[pose[:, -1] < 2.0, -1] = 0
obj['keypoints'] = pose.reshape((-1)).tolist()
obj['track_id'] = annots['tracks'][i]
obj['image_id'] = image_id
obj['category_id'] = 1 # person
objs.append(obj)
return objs
def _add_gt_annotations(self, entry, entry_id, annots):
if len(annots) > 0:
objs = self.convert_raw_predictions_to_objs(
annots[entry_id], entry['id'])
else:
ann_ids = self.COCO.getAnnIds(imgIds=entry['id'], iscrowd=None)
objs = self.COCO.loadAnns(ann_ids)
# Sanitize bboxes -- some are invalid
valid_objs = []
valid_segms = []
width = entry['width']
height = entry['height']
for obj in objs:
# crowd regions are RLE encoded and stored as dicts
if isinstance(obj['segmentation'], list):
# Valid polygons have >= 3 points, so require >= 6 coordinates
obj['segmentation'] = [
p for p in obj['segmentation'] if len(p) >= 6
]
if obj['area'] < cfg.TRAIN.GT_MIN_AREA:
continue
if 'ignore' in obj and obj['ignore'] == 1:
continue
# Convert form x1, y1, w, h to x1, y1, x2, y2
x1 = obj['bbox'][0]
y1 = obj['bbox'][1]
x2 = x1 + np.maximum(0., obj['bbox'][2] - 1.)
y2 = y1 + np.maximum(0., obj['bbox'][3] - 1.)
x1, y1, x2, y2 = box_utils.clip_xyxy_to_image(
x1, y1, x2, y2, height, width)
# Require non-zero seg area and more than 1x1 box size
if obj['area'] > 0 and x2 > x1 and y2 > y1:
obj['clean_bbox'] = [x1, y1, x2, y2]
valid_objs.append(obj)
valid_segms.append(obj['segmentation'])
num_valid_objs = len(valid_objs)
boxes = np.zeros((num_valid_objs, 4), dtype=entry['boxes'].dtype)
tracks = -np.ones((num_valid_objs, 1), dtype=entry['tracks'].dtype)
head_boxes = -np.ones((num_valid_objs, 4),
dtype=entry['head_boxes'].dtype)
gt_classes = np.zeros((num_valid_objs), dtype=entry['gt_classes'].dtype)
gt_overlaps = np.zeros(
(num_valid_objs, self.num_classes),
dtype=entry['gt_overlaps'].dtype)
seg_areas = np.zeros((num_valid_objs), dtype=entry['seg_areas'].dtype)
is_crowd = np.zeros((num_valid_objs), dtype=entry['is_crowd'].dtype)
box_to_gt_ind_map = np.zeros(
(num_valid_objs), dtype=entry['box_to_gt_ind_map'].dtype)
if self.keypoints is not None:
gt_keypoints = np.zeros(
(num_valid_objs, 3, self.num_keypoints),
dtype=entry['gt_keypoints'].dtype)
im_has_visible_keypoints = False
for ix, obj in enumerate(valid_objs):
cls = self.json_category_id_to_contiguous_id[obj['category_id']]
boxes[ix, :] = obj['clean_bbox']
if 'track_id' in obj:
tracks[ix, 0] = obj['track_id']
if 'head_box' in obj:
# NOTE: This box has NOT BEEN CLEANED, and NOT BEEN converted
# to (xmin, ymin, xmax, ymax). This is only here to be used
# in MPII evaluations
head_boxes[ix, :] = obj['head_box']
gt_classes[ix] = cls
seg_areas[ix] = obj['area']
is_crowd[ix] = obj['iscrowd']
box_to_gt_ind_map[ix] = ix
if self.keypoints is not None:
gt_keypoints[ix, :, :] = self._get_gt_keypoints(obj)
if np.sum(gt_keypoints[ix, 2, :]) > 0:
im_has_visible_keypoints = True
if obj['iscrowd']:
# Set overlap to -1 for all classes for crowd objects
# so they will be excluded during training
gt_overlaps[ix, :] = -1.0
else:
gt_overlaps[ix, cls] = 1.0
entry['boxes'] = np.append(entry['boxes'], boxes, axis=0)
entry['tracks'] = np.append(entry['tracks'], tracks, axis=0)
entry['head_boxes'] = np.append(entry['head_boxes'], head_boxes, axis=0)
entry['segms'].extend(valid_segms)
# To match the original implementation:
# entry['boxes'] = np.append(
# entry['boxes'], boxes.astype(np.int).astype(np.float), axis=0)
entry['gt_classes'] = np.append(entry['gt_classes'], gt_classes)
entry['seg_areas'] = np.append(entry['seg_areas'], seg_areas)
entry['gt_overlaps'] = np.append(
entry['gt_overlaps'].toarray(), gt_overlaps, axis=0)
entry['gt_overlaps'] = scipy.sparse.csr_matrix(entry['gt_overlaps'])
entry['is_crowd'] = np.append(entry['is_crowd'], is_crowd)
entry['box_to_gt_ind_map'] = np.append(
entry['box_to_gt_ind_map'], box_to_gt_ind_map)
if self.keypoints is not None:
entry['gt_keypoints'] = np.append(
entry['gt_keypoints'], gt_keypoints, axis=0)
entry['has_visible_keypoints'] = im_has_visible_keypoints
def _add_proposals_from_file(
self, roidb, proposal_file, min_proposal_size, top_k, crowd_thresh):
logger.info('Loading proposals from: {}'.format(proposal_file))
with open(proposal_file, 'r') as f:
proposals = pickle.load(f)
id_field = 'indexes' if 'indexes' in proposals else 'ids' # compat fix
_sort_proposals(proposals, id_field)
box_list = []
for i, entry in enumerate(roidb):
if i % 2500 == 0:
logger.info(' {:d}/{:d}'.format(i + 1, len(roidb)))
boxes = proposals['boxes'][i]
# Sanity check that these boxes are for the correct image id
assert entry['id'] == proposals[id_field][i]
# Remove duplicate boxes and very small boxes and then take top k
boxes = box_utils.clip_boxes_to_image(
boxes, entry['height'], entry['width'])
keep = box_utils.unique_boxes(boxes)
boxes = boxes[keep, :]
keep = box_utils.filter_small_boxes(boxes, min_proposal_size)
boxes = boxes[keep, :]
if top_k > 0:
boxes = boxes[:top_k, :]
box_list.append(boxes)
_merge_proposal_boxes_into_roidb(roidb, box_list)
if crowd_thresh > 0:
_filter_crowd_proposals(roidb, crowd_thresh)
def _init_keypoints(self, name=''):
self.keypoints = None
self.keypoint_flip_map = None
self.keypoints_to_id_map = None
self.num_keypoints = 0
# Thus far only the 'person' category has keypoints
if 'person' in self.category_to_id_map:
cat_info = self.COCO.loadCats([self.category_to_id_map['person']])
else:
return
# Check if the annotations contain keypoint data or not
if 'keypoints' in cat_info[0]:
keypoints = cat_info[0]['keypoints']
self.keypoints_to_id_map = dict(
zip(keypoints, range(len(keypoints))))
self.keypoints = keypoints
self.num_keypoints = len(keypoints)
if name.startswith('keypoints_coco'):
self.keypoint_flip_map = {
'left_eye': 'right_eye',
'left_ear': 'right_ear',
'left_shoulder': 'right_shoulder',
'left_elbow': 'right_elbow',
'left_wrist': 'right_wrist',
'left_hip': 'right_hip',
'left_knee': 'right_knee',
'left_ankle': 'right_ankle'}
else:
self.keypoint_flip_map = {
'left_shoulder': 'right_shoulder',
'left_elbow': 'right_elbow',
'left_wrist': 'right_wrist',
'left_hip': 'right_hip',
'left_knee': 'right_knee',
'left_ankle': 'right_ankle'}
def _get_gt_keypoints(self, obj):
if 'keypoints' not in obj:
return None
kp = np.array(obj['keypoints'])
x = kp[0::3] # 0-indexed x coordinates
y = kp[1::3] # 0-indexed y coordinates
# 0: not labeled; 1: labeled, not inside mask;
# 2: labeled and inside mask
v = kp[2::3]
num_keypoints = len(obj['keypoints']) / 3
assert num_keypoints == self.num_keypoints
gt_kps = np.ones((3, self.num_keypoints), dtype=np.int32)
for i in range(self.num_keypoints):
gt_kps[0, i] = x[i]
gt_kps[1, i] = y[i]
gt_kps[2, i] = v[i]
return gt_kps
def _split_roidb_frames(self, roidb):
# Config options
clips_per_video = cfg.VIDEO.DEFAULT_CLIPS_PER_VIDEO
clip_length = 1 # 1-frame clips
if self.annotations_info is not None:
clips_per_video = self.annotations_info['clips_per_video']
clip_length = self.annotations_info['clip_length']
entry_to_shard = _assign_shard_id_to_roidb(
roidb, self.annotations_info['num_splits'],
self.annotations_info['tot_vids'])
# For each video in roidb, split into a entry per-frame
new_roidb = []
new_annots = []
for entry_id, entry in enumerate(tqdm(roidb, desc='Splitting video->frames')):
assert 'nframes' in entry, 'Video dataset must have nframes'
# Get annotations, if possible
annots = {}
if self.annotations_info is not None:
annots = _read_weak_annotations(
entry_to_shard[entry_id],
data_dir=self.annotations_info['data_dir'],
det_file_name=self.annotations_info['det_file_name'])
assert(len(annots['boxes']) == entry['nframes'])
# roidb frame_ids are 1-indexed
already_added = {} # don't add same frame multiple times
step_size = max(entry['nframes'] // clips_per_video, 1)
for start_frame_id in range(1, entry['nframes'] + 2 - clip_length,
step_size):
for frame_id in range(start_frame_id, start_frame_id + clip_length):
if frame_id in already_added:
continue
new_entry = copy.deepcopy(entry)
new_entry['frame_id'] = frame_id
new_roidb.append(new_entry)
if len(annots) != 0:
new_annots.append({
# frame_id is 1-indexed
'boxes': annots['boxes'][frame_id - 1],
'poses': annots['poses'][frame_id - 1],
'tracks': annots['tracks'][frame_id - 1],
})
already_added[frame_id] = True
logger.info('New roidb size {}'.format(len(new_roidb)))
return new_roidb, new_annots
def _merge_proposal_boxes_into_roidb(roidb, box_list):
assert len(box_list) == len(roidb)
for i, entry in enumerate(roidb):
boxes = box_list[i]
num_boxes = boxes.shape[0]
gt_overlaps = np.zeros(
(num_boxes, entry['gt_overlaps'].shape[1]),
dtype=entry['gt_overlaps'].dtype)
box_to_gt_ind_map = -np.ones(
(num_boxes), dtype=entry['box_to_gt_ind_map'].dtype)
# Note: unlike in other places, here we intentionally include all gt
# rois, even ones marked as crowd. Boxes that overlap with crowds will
# be filtered out later (see: _filter_crowd_proposals).
gt_inds = np.where(entry['gt_classes'] > 0)[0]
if len(gt_inds) > 0:
gt_boxes = entry['boxes'][gt_inds, :]
gt_classes = entry['gt_classes'][gt_inds]
proposal_to_gt_overlaps = box_utils.bbox_overlaps(
boxes.astype(dtype=np.float32, copy=False),
gt_boxes.astype(dtype=np.float32, copy=False))
# Gt box that overlaps each input box the most
# (ties are broken arbitrarily by class order)
argmaxes = proposal_to_gt_overlaps.argmax(axis=1)
# Amount of that overlap
maxes = proposal_to_gt_overlaps.max(axis=1)
# Those boxes with non-zero overlap with gt boxes
I = np.where(maxes > 0)[0]
# Record max overlaps with the class of the appropriate gt box
gt_overlaps[I, gt_classes[argmaxes[I]]] = maxes[I]
box_to_gt_ind_map[I] = gt_inds[argmaxes[I]]
entry['boxes'] = np.append(
entry['boxes'],
boxes.astype(entry['boxes'].dtype, copy=False),
axis=0)
entry['gt_classes'] = np.append(
entry['gt_classes'],
np.zeros((num_boxes), dtype=entry['gt_classes'].dtype))
entry['seg_areas'] = np.append(
entry['seg_areas'],
np.zeros((num_boxes), dtype=entry['seg_areas'].dtype))
entry['gt_overlaps'] = np.append(
entry['gt_overlaps'].toarray(), gt_overlaps, axis=0)
entry['gt_overlaps'] = scipy.sparse.csr_matrix(entry['gt_overlaps'])
entry['is_crowd'] = np.append(
entry['is_crowd'],
np.zeros((num_boxes), dtype=entry['is_crowd'].dtype))
entry['box_to_gt_ind_map'] = np.append(
entry['box_to_gt_ind_map'],
box_to_gt_ind_map.astype(
entry['box_to_gt_ind_map'].dtype, copy=False))
def _filter_crowd_proposals(roidb, crowd_thresh):
"""Finds proposals that are inside crowd regions and marks them as
overlap = -1 with each ground-truth rois, which means they will be excluded
from training.
"""
for entry in roidb:
gt_overlaps = entry['gt_overlaps'].toarray()
crowd_inds = np.where(entry['is_crowd'] == 1)[0]
non_gt_inds = np.where(entry['gt_classes'] == 0)[0]
if len(crowd_inds) == 0 or len(non_gt_inds) == 0:
continue
crowd_boxes = box_utils.xyxy_to_xywh(entry['boxes'][crowd_inds, :])
non_gt_boxes = box_utils.xyxy_to_xywh(entry['boxes'][non_gt_inds, :])
iscrowd_flags = [int(True)] * len(crowd_inds)
ious = COCOmask.iou(non_gt_boxes, crowd_boxes, iscrowd_flags)
bad_inds = np.where(ious.max(axis=1) > crowd_thresh)[0]
gt_overlaps[non_gt_inds[bad_inds], :] = -1
entry['gt_overlaps'] = scipy.sparse.csr_matrix(gt_overlaps)
def _add_class_assignments(roidb):
for entry in roidb:
gt_overlaps = entry['gt_overlaps'].toarray()
# max overlap with gt over classes (columns)
max_overlaps = gt_overlaps.max(axis=1)
# gt class that had the max overlap
max_classes = gt_overlaps.argmax(axis=1)
entry['max_classes'] = max_classes
entry['max_overlaps'] = max_overlaps
# sanity checks
# if max overlap is 0, the class must be background (class 0)
zero_inds = np.where(max_overlaps == 0)[0]
assert all(max_classes[zero_inds] == 0)
# if max overlap > 0, the class must be a fg class (not class 0)
nonzero_inds = np.where(max_overlaps > 0)[0]
assert all(max_classes[nonzero_inds] != 0)
def _sort_proposals(proposals, id_field):
order = np.argsort(proposals[id_field])
fields_to_sort = ['boxes', id_field, 'scores']
for k in fields_to_sort:
proposals[k] = [proposals[k][i] for i in order]
def add_proposals(roidb, rois, scales):
"""Add proposal boxes (rois) to an roidb that has ground-truth annotations
but no proposals. If the proposals are not at the original image scale,
specify the scale factor that separate them in scales.
"""
box_list = []
for i in range(len(roidb)):
inv_im_scale = 1. / scales[i]
idx = np.where(rois[:, 0] == i)[0]
box_list.append(rois[idx, 1:] * inv_im_scale)
_merge_proposal_boxes_into_roidb(roidb, box_list)
# For historical consistency, not filter crowds (TODO(rbg): investigate)
# json_dataset._filter_crowd_proposals(roidb, cfg.TRAIN.CROWD_FILTER_THRESH)
_add_class_assignments(roidb)
def assign_shard_id_to_roidb(roidb, num_splits, tot_vids):
"""
Returns:
list with one element for each entry in roidb
(shard_dir_name,
(start_frame_id (0-indexed, included),
end_frame_id (0-indexed, not included)))
"""
shards = []
vids_per_job = int(math.ceil(tot_vids / num_splits))
last_proc = 0
for start_id in range(num_splits):
this_end_pos = min(last_proc + vids_per_job, tot_vids + 1)
this_outdir = '{0:05d}range{1}{2}'.format(
start_id, last_proc, this_end_pos)
# run through the entries that get assigned to this shard, and set
# what frames out of it belong to which video.
last_frame_proc = 0
for i in range(last_proc, min(this_end_pos, len(roidb))):
# start_id is included and last_proc is not, as happens in the
# ROIDB_SUBSET code
this_frame_proc = last_frame_proc + roidb[i]['nframes']
shards.append((
this_outdir, (last_frame_proc, this_frame_proc)))
last_frame_proc = this_frame_proc
last_proc = this_end_pos
return shards
def pickle_cached_load(fpath, cache):
if fpath in cache:
return cache[fpath]
with open(fpath, 'r') as fin:
data = pickle.load(fin)
cache.clear()
cache[fpath] = data
return data
@static_vars(weak_annot_cache={})
def _read_weak_annotations(shard_info, data_dir='',
det_file_name='detections.pkl',
fixed_str='test/kinetics_unlabeled_train/keypoint_rcnn'):
det_fpath = os.path.join(data_dir, shard_info[0], fixed_str, det_file_name)
data = pickle_cached_load(det_fpath, _read_weak_annotations.weak_annot_cache)
boxes = data['all_boxes'][1][shard_info[1][0]: shard_info[1][1]]
poses = data['all_keyps'][1][shard_info[1][0]: shard_info[1][1]]
tracks = data['all_tracks'][1][shard_info[1][0]: shard_info[1][1]]
assert(len(boxes) == len(poses))
assert(len(boxes) == len(tracks))
return {'boxes': boxes, 'poses': poses, 'tracks': tracks}
the '/home/amax/Documents/sangyi/DetectAndTrack/lib/datasets/data/PoseTrack/' is my image directory.
how can i fix it, i need anyone's help! thank you very muchi.
@rohitgirdhar
HI,
i have tried to run all the example codes like launch.py as well as tools/test_on_one_video.py.
Every time I am getting the same error.
Traceback (most recent call last):
File "tools/compute_tracks.py", line 20, in
from core.config import (
ImportError: cannot import name cfg_from_file
can anyone please help me ?
/home/kwduan2/sdb/anaconda2/envs/detect_and_track/bin/python: No module named peachpy.x86_64
confu-deps/NNPACK/CMakeFiles/nnpack.dir/build.make:61: recipe for target 'confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o' failed
make[2]: *** [confu-deps/NNPACK/src/x86_64-fma/2d-fourier-8x8.py.o] Error 1
CMakeFiles/Makefile2:205: recipe for target 'confu-deps/NNPACK/CMakeFiles/nnpack.dir/all' failed
make[1]: *** [confu-deps/NNPACK/CMakeFiles/nnpack.dir/all] Error 2
I tried to install DetectAndTrack from source, but failed, thus I turned to Dockerfile, and successfully installed caffe2 and DetectAndTrack. All the test passed until I the train module:python launch.py -c configs/video/2d_best/01_R101_best_hungarian.yaml -m train, and it saysAttributeError: Method AffineChannelNd is not a registered operator. Did you mean: [AffineChannel]. I investigated this problem and found that the lib/ops/affine_channel_nd_op.* need to be copied to caffe2/modules/detectron/. However, there is no detectron folder under the caffe2 installed by dockerfile, and then I git clone the detectron to the caffe2 folder. After I copy all the affine_channel_nd_op.* into the /detectron/ops/, and make, it occurred the error:/usr/local/caffe2_build/include/caffe2/utils/math.h:36:22: fatal error: Eigen/Core: No such file or directory. I found several solution for this error, but all of them need to revise the .cc files, for example, caffe2/core/context.cc file. But there is no such files in caffe2 using Dockerfile. How to deal with this problem?
The following command does not seem to work atm because of the absence of the best_model.pkl
file inside the 01_R101_best_hungarian.yaml
folder of pretrained_models
.
python launch.py \ --cfg configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml \ --mode test \ TEST.WEIGHTS pretrained_models/configs/video/2d_best/01_R101_best_hungarian.yaml/model_final.pkl
Do you suggest to use the 4-GPU one or can we expect a best_model.pkl
file ?
Thanks for the code and kudos on a very nice work!
What parts of this repository do i need to interface the tracking part with the results obtained running detectron on video frames?
Thanks
Where is run-net () and create_net() is located.(<built-in method run_net of PyCapsule object at 0x7f047a09cd20> & <built-in method create_net of PyCapsule object at 0x7f047a09ccf0>).
When run_net function gets called in the file "workspace.py" I am getting an exception and when I E
searched for the run_net method it's in "test loader.py" , but it's not entering there.RuntimeError:
**Exception is this:
RuntimeError: [enforce fail at conv_op_cudnn.cc:557] filter.dim32(i + 2) == kernel_[i]. 7 vs 1 Error from operator:
input: "gpu_0/data" input: "gpu_0/conv1_w" output: "gpu_0/conv1" name: "" type: "Conv" arg { name: "kernels" ints: 1 ints: 7 ints: 7 } arg { name: "exhaustive_search" i: 0 } arg { name: "order" s: "NCHW" } arg { name: "pads" ints: 0 ints: 3 ints: 3 ints: 0 ints: 3 ints: 3 } arg { name: "strides" ints: 1 ints: 2 ints: 2 } device_option { device_type: 1 cuda_gpu_id: 0 } engine: "CUDNN"**
When I follow the command of Testing pre-trained models
$ python launch.py \
--cfg configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml \
--mode test \
TEST.WEIGHTS pretrained_models/configs/video/2d_best/01_R101_best_hungarian.yaml/model_final.pkl
Error occurs
Done (t=0.54s)
creating index...
index created!
INFO subprocess.py: 53: detection range command 0: tools/test_net.py --range 0 1477 --cfg outputs/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/test/posetrack_v1.0_val/keypoint_rcnn/detection_range_config.yaml NUM_GPUS 1
INFO subprocess.py: 53: detection range command 1: tools/test_net.py --range 1477 2954 --cfg outputs/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/test/posetrack_v1.0_val/keypoint_rcnn/detection_range_config.yaml NUM_GPUS 1
INFO subprocess.py: 53: detection range command 2: tools/test_net.py --range 2954 4430 --cfg outputs/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/test/posetrack_v1.0_val/keypoint_rcnn/detection_range_config.yaml NUM_GPUS 1
INFO subprocess.py: 53: detection range command 3: tools/test_net.py --range 4430 5906 --cfg outputs/configs/video/2d_best/01_R101_best_hungarian-4GPU.yaml/test/posetrack_v1.0_val/keypoint_rcnn/detection_range_config.yaml NUM_GPUS 1
INFO subprocess.py: 80: # ---------------------------------------------------------------------------- #
INFO subprocess.py: 82: stdout of subprocess 0 with range [1, 1477]
INFO subprocess.py: 83: # ---------------------------------------------------------------------------- #
tools/test_net.py: line 10: Test a Fast R-CNN network on an image database.: command not found
tools/test_net.py: line 12: from: command not found
tools/test_net.py: line 13: from: command not found
tools/test_net.py: line 14: from: command not found
tools/test_net.py: line 15: from: command not found
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
tools/test_net.py: line 18: from: command not found
tools/test_net.py: line 21: from: command not found
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
import: unable to grab mouse `': Resource temporarily unavailable @ error/xwindow.c/XSelectWindow/9047.
tools/test_net.py: line 34: syntax error near unexpected token `FORMAT'
tools/test_net.py: line 34: `FORMAT = '%(levelname)s %(filename)s:%(lineno)4d: %(message)s''
Traceback (most recent call last):
File "tools/test_net.py", line 142, in <module>
main(ind_range=args.range, multi_gpu_testing=args.multi_gpu_testing)
File "tools/test_net.py", line 103, in main
engine.test_net_on_dataset(multi_gpu=multi_gpu_testing)
File "/data3/lilin/my_detect_and_track/DetectAndTrack/lib/core/test_engine.py", line 319, in test_net_on_dataset
num_images, output_dir)
File "/data3/lilin/my_detect_and_track/DetectAndTrack/lib/core/test_engine.py", line 284, in multi_gpu_test_net_on_dataset
'detection', num_images, binary, output_dir)
File "/data3/lilin/my_detect_and_track/DetectAndTrack/lib/utils/subprocess.py", line 67, in process_in_parallel
log_subprocess_output(i, p, output_dir, tag, start, end)
File "/data3/lilin/my_detect_and_track/DetectAndTrack/lib/utils/subprocess.py", line 97, in log_subprocess_output
assert ret == 0, 'Range subprocess failed (exit code: {})'.format(ret)
AssertionError: Range subprocess failed (exit code: 2)
I've googled a lot but could not find a answer.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.