jiyanggao / video-person-reid Goto Github PK

Video-based Person ReID Method Implementations on MARS

Python 100.00%

video-person-reid's Introduction

Video-Person-ReID

This is the code repository for our tech report "Revisiting Temporal Modeling for Video-based Person ReID": https://arxiv.org/abs/1805.02104. If you find this help your research, please cite

@article{gao2018revisiting,
  title={Revisiting Temporal Modeling for Video-based Person ReID},
  author={Gao, Jiyang and Nevatia, Ram},
  journal={arXiv preprint arXiv:1805.02104},
  year={2018}
}

Introduction

This repository contains PyTorch implementations of temporal modeling methods for video-based person reID. It is forked from deep-person-reid.. Based on that, I implement (1) video sampling strategy for training and testing, (2) temporal modeling methods including temporal pooling, temporal attention, RNN and 3D conv. The base loss function and basic training framework remain the same as deep-person-reid. PyTorch 0.3.1, Torchvision 0.2.0 and Python 2.7 is used.

Motivation

Although previous work proposed many temporal modeling methods and did extensive experiments, but it's still hard for us to have an "apple-to-apple" comparison across these methods. As the image-level feature extractor and loss function are not the same, which have large impact on the final performance. Thus, we want to test the representative methods under an uniform framework.

Dataset

All experiments are done on MARS, as it is the largest dataset available to date for video-based person reID. Please follow deep-person-reid to prepare the data. The instructions are copied here:

Create a directory named mars/ under data/.
Download dataset to data/mars/ from http://www.liangzheng.com.cn/Project/project_mars.html.
Extract bbox_train.zip and bbox_test.zip.
Download split information from https://github.com/liangzheng06/MARS-evaluation/tree/master/info and put info/ in data/mars (we want to follow the standard split in [8]). The data structure would look like:

mars/
    bbox_test/
    bbox_train/
    info/

Use -d mars when running the training code.

Usage

To train the model, please run

python main_video_person_reid.py --arch=resnet50tp

arch could be resnet50tp (Temporal Pooling), resnet50ta (Temporal Attention), resnet50rnn (RNN), resnet503d (3D conv). For 3D conv, I use the design and implementation from 3D-ResNets-PyTorch, just minor modification is done to fit the network into this person reID system.

In my experiments, I found that learning rate has a significant impact on the final performance. Here are the learning rates I used (may not be the best): 0.0003 for temporal pooling, 0.0003 for temporal attention, 0.0001 for RNN, 0.0001 for 3D conv.

Other detailed settings for different temporal modeling could be found in models/ResNet.py

Performance

Model	mAP	CMC-1	CMC-5	CMC-10	CMC-20
image-based	74.1	81.3	92.6	94.8	96.7
pooling	75.8	83.1	92.8	95.3	96.8
attention	76.7	83.3	93.8	96.0	97.4
rnn	73.9	81.6	92.8	94.7	96.3
3d conv	70.5	78.5	90.9	93.9	95.9

video-person-reid's People

Contributors

Stargazers

Watchers

Forkers

bityangke hxl1990 kikyou123 xiaoxying l1aoxingyu opencvfun xiangliu886 locussam longcw xiaoweihappy123 asuradayuci nanzhixionggit zysolanine rshivansh ganghu1993 helenligit lzj322 afcarl winroot hyzcn meimeiainaonao nianhao solomon1588 ltxue xxjjvxb linranran acewjh liyeuestc wanjinchang fdallaserra reeshark december-boy guofuzheng deep0learning kriszhou1 jianfweng gzzzzzzzzz swg209 yy19941128 hungminhsu amirunpri2018 whitesharkbrother xujing1022 robotjay supermousse yhl2018 zxr8192 tassadaryl hfutzzw wf825566323 lizhipengcs ligoml wangxinqi94 ai-driven tsungweihuang shuxjweb yuanwei0908 barbecacov yuange250 crazyusernova yeqi17 hell-to-heaven wangyingquan empireofkings without-peanut yangming-chaos wuyangfeng zetnim yf817 hopstone liuwenhaha sohuemily hsouporto achilles-s lxy1993 marenan currymew atonner anurag3 schliffen rollingccc cocosister lor-cat vanditg xujinglin aeril-ace ingeniousfrog noticeable mzq308734881 gengcode soulattacker zhangjiwei-japan dangweili bruinxiong borispf yangyangkiki txuanyu chrisdianru gaoziqiang xrosliang

video-person-reid's Issues

the initial file in models fold have a little mistakes

__factory = {
resnet50tp': ResNet50TP,
resnet50ta': ResNet50TP,
resnet50rnn': ResNet50RNN,
}
the third raw have a little mistakes

Evaluation Code Confusion relating to g_pid/q_pid mask for matches array

So I am trying to use resnet50ta for a task of my own. When running the algorithm to only evaluate on the Mars dataset, I print out gallery_ids and query_ids:

g_pids : [   0    0    0 ... 1496 1496 1500]
q_pids : [   2    2    2 ... 1496 1500 1500]

This is in accordance to the .mat split where there are query_IDX = 12179 and 12180 which in the track_test_info represent the 1500 tracklet.

The issue/confusion I have comes in the line 24 and 25 in eval_metrics.py:

remove = (g_pids[order] == q_pid) & (g_camids[order] == q_camid)
keep = np.invert(remove)

According to me we should not have the keep mask cause even though the pids of the tracklets are the same, they are different tracklets, thus have to be recombined/reid as the same id.

With commenting the masking lines the results increase, but I am not sure if this is correct or not, any clarification would be appreciated.

About the ResNet50TA network

Hi, I run the model ResNet50TA in Mars dataset. And I get the performance
CMC: 83.2, 93.5, 95.7, 97.1; map: 76.3.
But I find the attention seems not work, because the learned score for every frame is 0.25.

run resnet503d a bug

features = features.view(n, -1)#view()函数作用是将一个多行的Tensor，拼接成一行。

AttributeError: 'tuple' object has no attribute 'view'
Hi, Do you know how to solve it?

To run an error

When I run this code：python main_video_person_reid.py --arch=resnet50tp
Compared with the deep-person-reid project, I did not find the reason for error reporting.
Can you give me some suggestions.
Traceback (most recent call last):
File "main_video_person_reid.py", line 294, in
main()
File "main_video_person_reid.py", line 122, in main
pin_memory=pin_memory, drop_last=True,
File "/home/xiaowei/anaconda2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 437, in init
batch_sampler = BatchSampler(sampler, batch_size, drop_last)
File "/home/xiaowei/anaconda2/lib/python2.7/site-packages/torch/utils/data/sampler.py", line 124, in init
.format(sampler))
ValueError: sampler should be an instance of torch.utils.data.Sampler, but got sampler=<samplers.RandomIdentitySampler object at 0x7f87c3608b50>

license

would you pls add license?
thx

I have some questions about data_manager

Dear Gao:
Thank you very much your work.
When I use data_manager.py to deal with the MARS datasets, I found the query number is 626, and the gallery number is 622. Why the query number less than the gallery number?

把代码修改了在python3.5上跑，基础模型rank-1不超过50%？

how to know whether the model has converged or not?

when I trained the model, loss lingers around 1.2 all the time. I wonder how much loss will be after convergence when you are training the model.

when I used your model, the best performance is:

best-rank1: 0.76
mAP: 0.66

so, I worry about the model actually don't converge.

thank you so much.

loss log:

==> Epoch 151/800
Batch 5/12 Loss 1.151732 (1.184221)
Batch 10/12 Loss 1.185098 (1.208741)
==> Epoch 152/800
Batch 5/12 Loss 1.195779 (1.248034)
Batch 10/12 Loss 1.187773 (1.223771)
==> Epoch 153/800
Batch 5/12 Loss 1.199124 (1.226188)
Batch 10/12 Loss 1.244340 (1.216574)
==> Epoch 154/800
Batch 5/12 Loss 1.171001 (1.206451)
Batch 10/12 Loss 1.234953 (1.210964)
==> Epoch 155/800
Batch 5/12 Loss 1.171692 (1.175456)
Batch 10/12 Loss 1.286676 (1.226450)
==> Epoch 156/800
Batch 5/12 Loss 1.176779 (1.190393)
Batch 10/12 Loss 1.375999 (1.226623)
==> Epoch 157/800
Batch 5/12 Loss 1.252197 (1.201436)
Batch 10/12 Loss 1.162636 (1.229630)
==> Epoch 158/800
Batch 5/12 Loss 1.189308 (1.214950)
Batch 10/12 Loss 1.181550 (1.208582)
==> Epoch 159/800
Batch 5/12 Loss 1.128059 (1.146427)
Batch 10/12 Loss 1.189266 (1.164883)
==> Epoch 160/800
Batch 5/12 Loss 1.303839 (1.234521)
Batch 10/12 Loss 1.246772 (1.249348)
==> Epoch 161/800
Batch 5/12 Loss 1.279776 (1.200658)
Batch 10/12 Loss 1.450521 (1.241791)
==> Epoch 162/800
Batch 5/12 Loss 1.182487 (1.217111)
Batch 10/12 Loss 1.222181 (1.232863)
==> Epoch 163/800
Batch 5/12 Loss 1.215730 (1.204643)
Batch 10/12 Loss 1.201601 (1.217562)
==> Epoch 164/800
Batch 5/12 Loss 1.250196 (1.210316)
Batch 10/12 Loss 1.244857 (1.207420)
==> Epoch 165/800
Batch 5/12 Loss 1.213206 (1.162842)
Batch 10/12 Loss 1.271064 (1.197076)
==> Epoch 166/800
Batch 5/12 Loss 1.193753 (1.167730)
Batch 10/12 Loss 1.221726 (1.206353)
==> Epoch 167/800
Batch 5/12 Loss 1.202258 (1.226315)
Batch 10/12 Loss 1.281449 (1.228988)
==> Epoch 168/800
Batch 5/12 Loss 1.234186 (1.282551)
Batch 10/12 Loss 1.286633 (1.289782)
==> Epoch 169/800
Batch 5/12 Loss 1.176978 (1.193318)
Batch 10/12 Loss 1.212864 (1.219412)
==> Epoch 170/800
Batch 5/12 Loss 1.220609 (1.195325)
Batch 10/12 Loss 1.191236 (1.221831)
==> Epoch 171/800
Batch 5/12 Loss 1.377638 (1.262232)
Batch 10/12 Loss 1.282985 (1.233217)
==> Epoch 172/800
Batch 5/12 Loss 1.161230 (1.185284)
Batch 10/12 Loss 1.184502 (1.199786)
==> Epoch 173/800
Batch 5/12 Loss 1.168909 (1.182812)
Batch 10/12 Loss 1.270397 (1.197942)
==> Epoch 174/800
Batch 5/12 Loss 1.204447 (1.163887)
Batch 10/12 Loss 1.176649 (1.187690)
==> Epoch 175/800
Batch 5/12 Loss 1.198594 (1.184971)
Batch 10/12 Loss 1.302678 (1.217098)
==> Epoch 176/800
Batch 5/12 Loss 1.255241 (1.176709)
Batch 10/12 Loss 1.314871 (1.217345)
==> Epoch 177/800
Batch 5/12 Loss 1.457606 (1.255013)
Batch 10/12 Loss 1.256773 (1.233927)
==> Epoch 178/800
Batch 5/12 Loss 1.180098 (1.210582)
Batch 10/12 Loss 1.198249 (1.219975)
==> Epoch 179/800
Batch 5/12 Loss 1.415169 (1.233042)
Batch 10/12 Loss 1.247434 (1.268601)
==> Epoch 180/800
Batch 5/12 Loss 1.294599 (1.236518)
Batch 10/12 Loss 1.276755 (1.259600)
==> Epoch 181/800
Batch 5/12 Loss 1.345576 (1.215330)
Batch 10/12 Loss 1.286063 (1.239036)
==> Epoch 182/800
Batch 5/12 Loss 1.205686 (1.186106)
Batch 10/12 Loss 1.180740 (1.196601)
==> Epoch 183/800
Batch 5/12 Loss 1.212626 (1.198935)
Batch 10/12 Loss 1.218805 (1.222616)
==> Epoch 184/800
Batch 5/12 Loss 1.262955 (1.225434)
Batch 10/12 Loss 1.270376 (1.229889)
==> Epoch 185/800
Batch 5/12 Loss 1.184287 (1.197745)
Batch 10/12 Loss 1.258302 (1.225032)
==> Epoch 186/800
Batch 5/12 Loss 1.210953 (1.191885)
Batch 10/12 Loss 1.270131 (1.197497)
==> Epoch 187/800
Batch 5/12 Loss 1.239055 (1.209134)
Batch 10/12 Loss 1.215639 (1.213043)
==> Epoch 188/800
Batch 5/12 Loss 1.246695 (1.194201)
Batch 10/12 Loss 1.292399 (1.216818)
==> Epoch 189/800
Batch 5/12 Loss 1.197155 (1.217120)
Batch 10/12 Loss 1.179686 (1.225153)
==> Epoch 190/800
Batch 5/12 Loss 1.154700 (1.212153)
Batch 10/12 Loss 1.190199 (1.214623)
==> Epoch 191/800
Batch 5/12 Loss 1.315284 (1.203522)
Batch 10/12 Loss 1.280430 (1.217727)
==> Epoch 192/800
Batch 5/12 Loss 1.147254 (1.190746)
Batch 10/12 Loss 1.161241 (1.193478)
==> Epoch 193/800
Batch 5/12 Loss 1.260765 (1.196339)
Batch 10/12 Loss 1.136279 (1.181638)
==> Epoch 194/800
Batch 5/12 Loss 1.167764 (1.230083)
Batch 10/12 Loss 1.203083 (1.219795)
==> Epoch 195/800
Batch 5/12 Loss 1.213726 (1.254149)
Batch 10/12 Loss 1.210874 (1.226763)
==> Epoch 196/800
Batch 5/12 Loss 1.157766 (1.199533)
Batch 10/12 Loss 1.231522 (1.201280)
==> Epoch 197/800
Batch 5/12 Loss 1.145700 (1.159924)
Batch 10/12 Loss 1.217924 (1.186239)
==> Epoch 198/800
Batch 5/12 Loss 1.230108 (1.180395)
Batch 10/12 Loss 1.216664 (1.208750)
==> Epoch 199/800
Batch 5/12 Loss 1.188747 (1.171446)
Batch 10/12 Loss 1.129958 (1.195133)
==> Epoch 200/800

the pretrained models: 3dconv-person-reid/pretrained_models/resnet-50-kinetics.pth

3dconv-person-reid/pretrained_models/resnet-50-kinetics.pth
can you provide the models using baiduyunpan?

out of memory problem

Hi,

Thanks for your work. I got out of memory problem during test. I then printed out the shapes of images and found that it crashed when I got imgs.shape=[65, 4, 3, .., ..] and it seems like I need to process 65 * 4 images on the GPU card in one shot. I guess this is a problem even I've got 12GB GPU memory.

So I am wondering how you deal with this problem? Did you just use a gpu card with enough memory or you did anything else? Please let me know and much appreciate your work.

Update:
I was using torch 0.4 , python 3.6. I have switched to torch 0.3 and python 2.7 and so far it works fine.

Thanks.

training

when i run the code ,i have the problem:
Traceback (most recent call last):
File "main_video_person_reid.py", line 293, in
main()
File "main_video_person_reid.py", line 118, in main
pin_memory=pin_memory, drop_last=True,
File "/home/jiayan/anaconda3/envs/py35/lib/python3.5/site-packages/torch/utils/data/dataloader.py", line 437, in init
batch_sampler = BatchSampler(sampler, batch_size, drop_last)
File "/home/jiayan/anaconda3/envs/py35/lib/python3.5/site-packages/torch/utils/data/sampler.py", line 124, in init
.format(sampler))
ValueError: sampler should be an instance of torch.utils.data.Sampler, but got sampler=<samplers.RandomIdentitySampler object at 0x7fb136700f60>

i don't know where is the prolem. thank you very much! @jiyanggao

Different Way to implement Triplet loss

Hi,
Can you please explain the way you implement triplet loss a little bit. I implement triplet loss like that. But I do not understand your implementation.
`class TripletLoss(nn.Module):
"""
Triplet loss
Takes embeddings of an anchor sample, a positive sample and a negative sample
"""

def __init__(self, margin):
    super(TripletLoss, self).__init__()
    self.margin = margin

def forward(self, anchor, positive, negative, size_average=True):
    distance_positive = (anchor - positive).pow(2).sum(1)  # .pow(.5)
    distance_negative = (anchor - negative).pow(2).sum(1)  # .pow(.5)
    losses = F.relu(distance_positive - distance_negative + self.margin)
    return losses.mean() if size_average else losses.sum()

Bug in mars split

Look like I found the problem in MARS dataset split. For some queries, I can't found right answers in the gallery, (and vice versa). Does anyone have the same problem?

=====================================

import numpy as np
from scipy.io import loadmat

q = loadmat('/media/re-id/datasets/mars/info/query_IDX.mat')['query_IDX'][0]
t = loadmat('/media/re-id/datasets/mars/info/tracks_test_info.mat')['track_test_info']

query_inds = q - 1
gallery_inds = [i for i in range(len(t)) if i not in query_inds]

query_pids = t[query_inds , 2]
gallery_pids = t[gallery_inds, 2]

print('This person ids only in query')
print(np.setdiff1d(query_pids, gallery_pids))
print()
print('This pids only in gallery')
print(np.setdiff1d(gallery_pids, query_pids))

============================================

This person ids only in query
[ 6 152 436 454 640 782 944 1062 1138 1140 1146 1172 1252 1444]

This pids only in gallery
[ -1 0 12 14 154 166 982 1034 1104 1354]

GPU Out of Memory

I made some changes on original codes to make it compatible with torch 1.0.1 and python 3.6. However, I encounter gpu out of memory problem during testing process. Does anyone have the same problem? And is it caused by version difference? I used 1080ti with 11GB memory.

When running command add --htri-only encount this problem

When I run the command : python main_video_person_reid.py --htri-only
got the error below. Could you please help me out?

 File "/home/swg/Re-id/Video_Person_Reid/losses.py", line 81, in forward
    dist_ap = torch.cat(dist_ap)
RuntimeError: zero-dimensional tensor (at position 0) cannot be concatenated
``

你好，请问怎么在自己数据集上训练，我有几段视频，标注好了行人框的位置后，怎么训练呢

test-batch 是只能设置为1 吗？

请问test-batch 是只能设置为1 吗？这样测试的时候太慢了，如果test-batch设置为别的数值的话，
queryloader = DataLoader(
VideoDataset(dataset.query, seq_len=args.seq_len, sample='dense', transform=transform_test),
batch_size=args.test_batch, shuffle=False, num_workers=args.workers,
pin_memory=pin_memory, drop_last=False,
)
就会报错，stack expects each tensor to be equal size，but got [10,4,3,256,128] at entry 0 and [25,4,3,256,128] at entry 1.
只有test-batch 为1的时候才能运行成功，请问大佬，有什么方法可以使test-batch 设置的大一些呢

About exit problem

Why when I run this code, alwarys appears "Process finished with exit code -1"

How to sample part of data from the Mars dataset?

I use the resnetrnn model, but after 800 epoch ,I got a bad result, mAP：19% ， rank-1：27.2%，rank-5：46.5%，rank-10：56.0%，rank-20：65.1% .So I want to cut down the mars dataset volumn,
to try to figure out why , but I don't konw how to make it? could you help me ?

possible to share pre-trained model?

Hi,

Thank you for sharing the code! But is there any chance that you can share pre-trained models here? It's a bit tricky to reproduce the results. It would be very much appreciated. Thanks again!

what environment did you run the program?

Thank you for your work
What environment did you run the program?
Why do I use 24 GB of GPU memory and still have an out-of-memory exception when I run test。
RuntimeError: CUDA out of memory.

About result

When using temporal polling，map is just 66.4%

bug in running resnet50rnn

Traceback (most recent call last):
File "main_video_person_reid.py", line 290, in
main()
File "main_video_person_reid.py", line 158, in main
model = nn.DataParallel(model).cuda()
File "/home/wjw/opt/anaconda3/envs/pytorch3/lib/python2.7/site-packages/torch/nn/modules/module.py", line 216, in cuda
return self._apply(lambda t: t.cuda(device))
File "/home/wjw/opt/anaconda3/envs/pytorch3/lib/python2.7/site-packages/torch/nn/modules/module.py", line 146, in _apply
module._apply(fn)
File "/home/wjw/opt/anaconda3/envs/pytorch3/lib/python2.7/site-packages/torch/nn/modules/module.py", line 146, in _apply
module._apply(fn)
File "/home/wjw/opt/anaconda3/envs/pytorch3/lib/python2.7/site-packages/torch/nn/modules/rnn.py", line 123, in _apply
self.flatten_parameters()
File "/home/wjw/opt/anaconda3/envs/pytorch3/lib/python2.7/site-packages/torch/nn/modules/rnn.py", line 111, in flatten_parameters
params = rnn.get_parameters(fn, handle, fn.weight_buf)
File "/home/wjw/opt/anaconda3/envs/pytorch3/lib/python2.7/site-packages/torch/backends/cudnn/rnn.py", line 165, in get_parameters
assert filter_dim_a.prod() == filter_dim_a[0]
AssertionError