alanlukezic / d3s Goto Github PK

It seems that pixel-level tracking has become a promising and dominant area in visual object tracking !

Dear @alanlukezic , I'm really interested in your wonderful work. However, where is the GEM part of the model implemented? From the ltr folder, it seems "test_dist" is feed in directly from data.

I find this code:"dist_map = self.create_dist(init_patch_crop.shape[0], init_patch_crop.shape[1])",but is that the "The target localisation channel (L)" mentioned in the paper?
In segm_net.py the code :
" segm_layers = torch.cat((torch.unsqueeze(pred_sm[:, :, :, 0], dim=1),
torch.unsqueeze(pred_pos, dim=1),
dist), dim=1)", It looks like this "The target localisation channel (L)" is fixed as the code "self.dist_map = dist_map",has the "self.dist_map" changed?and where ?
thanks~

File "/opt/research2/d3s/pytracking/tracker/segm/segm.py", line 96, in initialize
feat_max_stride = max(self.params.features_filter.stride())
File "/opt/research2/d3s/pytracking/features/extractor.py", line 66, in stride
return torch.Tensor(TensorList([f.stride() for f in self.features if self._return_feature(f)]).unroll())
TypeError: __cuda_array_interface__ must be a dict

I encountered the above problem while running your code, can you help to solve it？

Is there a basic function given it a picture and a first_state and then giving it the next pictures of a sequence?
I feel like the rabbit hole is too deep and I cannot use this project.
Or how should I use it in a live video feed where the pictures are coming from a RPi to my GPU server?

I like your project very much. But I test your tracker using pysot toolkit. The average eao ( 10 times )is around 0.46 which is lower than 0.489. I wonder if there are the differences of hardware or toolkit or not affectcting the results. Which toolkit do you use to evaluate your tracker?

Hi,

I am sorry if this issue of mine looks naive, I am just getting started with this.
When I type the code for running the tracker: python run_tracker.py
why the result is tracking,not segmentation?
Do some of you know how to address this issue?

Thank you in advance,
CoisiniZhang

I am confused why it happens
I set right path in votdataset,but when i run python run_tracker.py segm default_params vot

Tracker: segm default_params None , Sequence: agility
OpenCV(4.5.1) /tmp/pip-req-build-7m_g9lbm/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Tracker: segm default_params None , Sequence: ants1
OpenCV(4.5.1) /tmp/pip-req-build-7m_g9lbm/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Tracker: segm default_params None , Sequence: ball2
OpenCV(4.5.1) /tmp/pip-req-build-7m_g9lbm/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Tracker: segm default_params None , Sequence: ball3
OpenCV(4.5.1) /tmp/pip-req-build-7m_g9lbm/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'
.......
Done

How can I solve it?

Is there anyone who successfully using d3s on NX?

/home/bfj/d3s/pytracking/../pytracking/evaluation/vot18dataset.py:54: FutureWarning: The squeeze argument has been deprecated and will be removed in a future version. Append .squeeze("columns") to the call to squeeze.

sequence_list = pandas.read_csv(list_path, header=None, squeeze=True).values.tolist()
Tracker: segm default_params None , Sequence: ants1
/home/bfj/anaconda3/envs/pytracking/lib/python3.10/site-packages/torchvision/models/_utils.py:252: UserWarning: Accessing the model URLs via the internal dictionary of the module is deprecated since 0.13 and may be removed in the future. Please access them via the appropriate Weights Enum instead.
warnings.warn(
Traceback (most recent call last):
File "/home/bfj/d3s/pytracking/run_tracker.py", line 84, in
main()
File "/home/bfj/d3s/pytracking/run_tracker.py", line 80, in main
run_tracker(args.tracker_name, args.tracker_param, args.runid, args.dataset, args.sequence, args.debug, args.threads)
File "/home/bfj/d3s/pytracking/run_tracker.py", line 65, in run_tracker
run_dataset(dataset, trackers, debug, threads)
File "/home/bfj/d3s/pytracking/../pytracking/evaluation/running.py", line 54, in run_dataset
run_sequence(seq, tracker_info, debug=debug)
File "/home/bfj/d3s/pytracking/../pytracking/evaluation/running.py", line 21, in run_sequence
tracked_bb, exec_times = tracker.run(seq, debug=debug)
File "/home/bfj/d3s/pytracking/../pytracking/evaluation/tracker.py", line 58, in run
output_bb, execution_times = tracker.track_sequence(seq)
File "/home/bfj/d3s/pytracking/../pytracking/tracker/base/basetracker.py", line 35, in track_sequence
self.initialize(image, sequence.init_state)
File "/home/bfj/d3s/pytracking/../pytracking/tracker/segm/segm.py", line 96, in initialize
feat_max_stride = max(self.params.features_filter.stride())
File "/home/bfj/d3s/pytracking/../pytracking/features/extractor.py", line 66, in stride
return torch.Tensor(TensorList([f.stride() for f in self.features if self._return_feature(f)]).unroll())
TypeError: expected Tensor as argument 0, but got TensorList

Hi @alanlukezic Thank you for your work.
When training d3s, I don't know the meaning of rectangles.zip you provide us.
Taking groundtruth-1.txt for example, does each row means [xmin,ymin,weight,height] of object1 in that sequence?
Look forward to your reply!

I set ATOM's "use_gpu" as False, but I meet "TypeError: __cuda_array_interface__ must be a dict"

Dear @alanlukezic , the GEM module is the DCF filter which helps the model track an object in addition to segmenting it. What do you think of DCF filter vs a sliding window cross correlation template matching layer as done in some works like SiamFC? I've been trying to use the DCF layer for tracking but don't quite understand how to use it. There is a DCFNet implementation by Foolwood but it's not clear how to use it.

The official website of VOS dataset has no access to download. Is there any other way to obtain VOS training dataset？Thanks！

I tried the link of https://youtube-vos.org/dataset/ to download VOS2018
but it's not available to download the dataset. Could you please share me the dropbox link or google drive link?

Does anyone able to download Pre-trained model. I am not able to do, if possible can anyone share please.
Thanks in advance!

hello，thanks for your excellent job! I want to know whether the code can be trained on YouTube-VOS and then evaluate on DAVIS for VOS task，and how to implement it？Looking forward to your reply.

Please help me. Thank you.
Experiment baseline
Tracker D3S
Sequence ants1
Repetition 1
Repetition 2
Repetition 3
Repetition 4
Repetition 5
Repetition 6
Repetition 7
Repetition 8
Repetition 9
Repetition 10
Repetition 11
Repetition 12
Repetition 13
Repetition 14
Repetition 15
Sequence ants3
Repetition 1
Repetition 2
Repetition 3
Repetition 4
Repetition 5
Repetition 6
Repetition 7
Repetition 8
Repetition 9
Repetition 10
Repetition 11
Tracker execution interrupted: Unable to establish connection.

When I run the command： run_tracker.py segm default_params --dataset otb --debug 0, the error that No matching checkpoint file found will occur, how do I solve it? thanks please

Hi,

I am sorry if this issue of mine looks naive, I am just getting started with this.

When I type the code for running the tracker: python run_tracker.py segm default_params --dataset vot18 --sequence \media\evalli\HD_EUGI\Resolve_Videos\Elly_coreography.mp4 --debug 1, an error occurs, saying:

FileNotFoundError: [Errno 2] No such file or directory: '/home/evalli/d3s/pytracking/evaluation/list.txt'

I think this is related to the vot18-path selection in 'local.py', because every time I change that path, the error shows it, adding the 'list.txt' file who looks like missing or non-existing.

Do some of you know how to address this issue? Also, where is the VOT18 dataset located?

Thank you in advance,
Eugenio

I find your code has a projection matrix. But I don't know what function it has? and I even can not find where the network structure is.

Hi there, It will really kind if you could help me with this issue.

The Vos.py file looks for groundtruth-1.txt file. When I downloaded the VOS 2018 dataset, I didn't found the groundtruth-1.txt file in annotations directory. So while I run the code python run_training.py, I do get this error.
So kindly tell me actually where am I getting wrong.

Thanks in advance

Thanks for your brilliant work.I wonder how to draw picture like this in your paper,can you give me some advise? Thank you very much.

And also, can you give the parameter you use to test on got-10k? The result I get is inferior to yours.

when i run python run_tracker.py segm default_params

Traceback (most recent call last):
File "../pytracking/evaluation/otbdataset.py", line 60, in _construct_sequence
ground_truth_rect = np.loadtxt(str(anno_path), dtype=np.float64)
File "/home/chenzhe/anaconda3/envs/D3Spytracking/lib/python3.7/site-packages/numpy/lib/npyio.py", line 1146, in loadtxt
for x in read_data(_loadtxt_chunksize):
File "/home/chenzhe/anaconda3/envs/D3Spytracking/lib/python3.7/site-packages/numpy/lib/npyio.py", line 997, in read_data
items = [conv(val) for (conv, val) in zip(converters, vals)]
File "/home/chenzhe/anaconda3/envs/D3Spytracking/lib/python3.7/site-packages/numpy/lib/npyio.py", line 997, in
items = [conv(val) for (conv, val) in zip(converters, vals)]
File "/home/chenzhe/anaconda3/envs/D3Spytracking/lib/python3.7/site-packages/numpy/lib/npyio.py", line 734, in floatconv
return float(x)
ValueError: could not convert string to float: '250,168,106,105'

How can i solve it?

Hello, I used the trained model to test on THE GOT-10K, could you please tell me how I can get the visual effect of box and mask displayed at the same time? Where do you need to set the parameters? When I set debug = 1, all I can see is a visual display of the box.

I test your tracker using pysot toolkit.

After downloading the pre-training file from the given link, it is corrupted.
Please check it, thank you!

Thanks for your brilliant work.I have trouble understanding the refinement pathway.The input F,P focus on the object,what is the consideration of L which has higher response in the background?And it's really confusing if L has higher response in the foreground,after training,tracker behave worser than yours.

I have clone your code and vos dataset(version2018), but I came across a run error.
It show me that my dataset don't have a file in named 'groundtruth-obj_id.txt', in each seqence in youtube-vos dataset. My dataset have some wrong or miss some files ?

def _read_anno(self, seq_path, obj_id):
anno_file = os.path.join(seq_path, "groundtruth-%s.txt" % obj_id)
gt = pandas.read_csv(anno_file, delimiter=',', header=None, dtype=np.float32, na_filter=False, low_memory=False).values
return torch.tensor(gt)

Traceback (most recent call last):
File "run_tracker.py", line 84, in
main()
File "run_tracker.py", line 80, in main
run_tracker(args.tracker_name, args.tracker_param, args.runid, args.dataset, args.sequence, args.debug, args.threads)
File "run_tracker.py", line 65, in run_tracker
run_dataset(dataset, trackers, debug, threads)
File "../pytracking/evaluation/running.py", line 54, in run_dataset
run_sequence(seq, tracker_info, debug=debug)
File "../pytracking/evaluation/running.py", line 21, in run_sequence
tracked_bb, exec_times = tracker.run(seq, debug=debug)
File "../pytracking/evaluation/tracker.py", line 58, in run
output_bb, execution_times = tracker.track_sequence(seq)
File "../pytracking/tracker/base/basetracker.py", line 35, in track_sequence
self.initialize(image, sequence.init_state)
File "../pytracking/tracker/segm/segm.py", line 160, in initialize
self.init_optimization(train_x, init_y)
File "../pytracking/tracker/segm/segm.py", line 215, in init_optimization
self.joint_optimizer.run(self.params.init_CG_iter // self.params.init_GN_iter, self.params.init_GN_iter)
File "../pytracking/libs/optimization.py", line 345, in run
self.run_GN_iter(cg_iter)
File "../pytracking/libs/optimization.py", line 390, in run_GN_iter
delta_x, res = self.run_CG(num_cg_iter, eps=self.cg_eps)
File "../pytracking/libs/optimization.py", line 127, in run_CG
q = self.A(self.p)
File "../pytracking/libs/optimization.py", line 400, in A
dfdx_x = torch.autograd.grad(self.dfdxt_g, self.g, x, retain_graph=True)
File "/home/xl/anaconda3/envs/pytracking/lib/python3.7/site-packages/torch/autograd/init.py", line 157, in grad
inputs, allow_unused)

when i run 'python run_tracker.py segm default_params --dataset vot18 --debug 1,the error occurred,'Is there anybody have the same issue

Why is "pytracking/evaluation/local.py" not in the downloaded file?thanks

Hello, I made the following error during training. The data set I used was VOS:

FileNotFoundError: [Errno 2] File /media/panxiang/Data/train_dataset/vos/train/Annotations/4918d10ff0/groundtruth-2.txt does not exist: '/media/panxiang/Data/train_dataset/vos/train/Annotations/4918d10ff0/groundtruth-2.txt'

I found that there is no groundtruth. TXT file under the data set I downloaded. Could you please tell me where this file was generated?

Sorry to bother but I want to know which .py in the code is the implementation of GIM module. Thanks in advance.

Traceback (most recent call last):
File "run_video.py", line 36, in
main()
File "run_video.py", line 32, in main
run_video(args.tracker_name, args.tracker_param,args.videofile, args.optional_box, args.debug)
File "run_video.py", line 20, in run_video
tracker.run_video(videofilepath=videofile, optional_box=optional_box, debug=debug)
File "../pytracking/evaluation/tracker.py", line 77, in run_video
tracker.track_videofile(videofilepath, optional_box)
File "../pytracking/tracker/base/basetracker.py", line 91, in track_videofile
self.initialize(frame, init_state)
File "../pytracking/tracker/segm/segm.py", line 160, in initialize
self.init_optimization(train_x, init_y)
File "../pytracking/tracker/segm/segm.py", line 215, in init_optimization
self.joint_optimizer.run(self.params.init_CG_iter // self.params.init_GN_iter, self.params.init_GN_iter)
File "../pytracking/libs/optimization.py", line 345, in run
self.run_GN_iter(cg_iter)
File "../pytracking/libs/optimization.py", line 390, in run_GN_iter
delta_x, res = self.run_CG(num_cg_iter, eps=self.cg_eps)
File "../pytracking/libs/optimization.py", line 127, in run_CG
q = self.A(self.p)
File "../pytracking/libs/optimization.py", line 400, in A
dfdx_x = torch.autograd.grad(self.dfdxt_g, self.g, x, retain_graph=True)
File "/home/chenzhe/anaconda3/envs/D3Spytracking/lib/python3.7/site-packages/torch/autograd/init.py", line 157, in grad
inputs, allow_unused)
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.

I use the same cuda version with Dimp(pytracking)，it works in dimp but error here，how can i solve it?

Could the author provide the raw results of D3S paper?

Hi, D3S is an excellent work and I'm very interesting in the performance of D3Sv2 on VOT2021 benchmark. I found that the scorce codes of D3S and D3Sv2 on the VOT official web are the same. I wonder when d3sv2 will be open source? I'm looking forward to it.

Thank you for your great work, currently, I want to play around with it but all the hyperlinks in 'README.md' seens like already have expired. check

1. https://data.vicos.si/alanl/d3s/SegmNet.pth.tar
2. https://data.vicos.si/alanl/d3s/rectangles.zip

Any updates will be very appreciated, thank you in advance.

when i run python run_tracker.py segm default_params --dataset vot18 --sequence ball1 --debug 1
i got an error that

(pytracking) E:\master\Tracking\VOT\python vot\Trackers\d3s-master(1)\pytracking>python run_tracker.py segm default_params --dataset vot18 --sequence ball1 --debug 1
Traceback (most recent call last):
File "run_tracker.py", line 84, in
main()
File "run_tracker.py", line 80, in main
run_tracker(args.tracker_name, args.tracker_param, args.runid, args.dataset, args.sequence, args.debug, args.threads)
File "run_tracker.py", line 44, in run_tracker
dataset = VOT18Dataset()
File "..\pytracking\evaluation\vot18dataset.py", line 7, in VOT18Dataset
return VOTDatasetClass().get_sequence_list()
File "..\pytracking\evaluation\vot18dataset.py", line 24, in init
self.sequence_list = self._get_sequence_list()
File "..\pytracking\evaluation\vot18dataset.py", line 54, in _get_sequence_list
sequence_list = pandas.read_csv(list_path, header=None, squeeze=True).values.tolist()
File "C:\Users\Ahmed Osama\anaconda3\envs\pytracking\lib\site-packages\pandas\io\parsers.py", line 686, in read_csv
return _read(filepath_or_buffer, kwds)
File "C:\Users\Ahmed Osama\anaconda3\envs\pytracking\lib\site-packages\pandas\io\parsers.py", line 452, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "C:\Users\Ahmed Osama\anaconda3\envs\pytracking\lib\site-packages\pandas\io\parsers.py", line 946, in init
self._make_engine(self.engine)
File "C:\Users\Ahmed Osama\anaconda3\envs\pytracking\lib\site-packages\pandas\io\parsers.py", line 1178, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "C:\Users\Ahmed Osama\anaconda3\envs\pytracking\lib\site-packages\pandas\io\parsers.py", line 2008, in init
self._reader = parsers.TextReader(src, **kwds)
File "pandas_libs\parsers.pyx", line 382, in pandas._libs.parsers.TextReader.cinit
File "pandas_libs\parsers.pyx", line 674, in pandas._libs.parsers.TextReader._setup_parser_source
OSError: [Errno 22] Invalid argument: 'E:\master\Tracking\VOT\python vot\x0bot-workspace/list.txt'

can you help me to run this tracker
im using python 3.8
windows 10
anaconda sw

--squence is ?????

File "../pytracking/libs/tensorlist.py", line 97
return TensorList([e1 @ e2 for e1, e2 in zip(self, other)])
^
SyntaxError: invalid syntax

Is it because I used Cuda10？

Hi, I think d3s is an excellent work which combined both tracking and segmentation. I like it very much. However, when I tested the pretrained model on davis2016, I got results as follow.

it's not as same as the paper.

I am confused. Can you give me some advice?

I found your project very interesting. Your paper says that it performs tracking and segmentation. I test if for tracking but could not run it for segmentation. Could you please help me to run your project for video segmentation? For example to run over DAVIS or YouTube-VOS datasets. I am grateful for your help.

a strange eroor happened when i run run_tracker on vot18.I want to know how to fix it?

Hi,
I cant find segm_predictor() function implementation in this repository.
Would please tell me where it is implemented ? and why it needs train_feat_segm as input argument ?
Thanks

Hi,
I tried to use D3S but I couldn't run it, I followed all the required steps and I found these errors.

First of all I downloaded the d3s repository with git clone in the cluster that I use. Then I installed the dependencies using the install.sh file.
I set in the default_params.py file the path for the segmentation mask and the path for the pre-trained network.

I installed the vot-toolkit using this guide: https://www.votchallenge.net/howto/tutorial_python.html

I created the workspace itself inside the pytracking folder, downloading the vot2018 dataset.

in the evaluation folder I modified the file environment setting the path for the dataset generating the file local.py

I activated the conda environment pytracking
I run the script with the command:
python run_tracker.py segm default_params --dataset vot18 --sequence ball1 --debug 1

That's the mistake that detects me:

/home/antoniodimarino/.conda/envs/pytracking/lib/python3.7/site-packages/torch/nn/functional.py:2539: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
  "See the documentation of nn.Upsample for details.".format(mode))
/home/antoniodimarino/.conda/envs/pytracking/lib/python3.7/site-packages/torch/nn/functional.py:2457: UserWarning: nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.
  warnings.warn("nn.functional.upsample is deprecated. Use nn.functional.interpolate instead.")
Traceback (most recent call last):
  File "run_tracker.py", line 84, in <module>
    main()
  File "run_tracker.py", line 80, in main
    run_tracker(args.tracker_name, args.tracker_param, args.runid, args.dataset, args.sequence, args.debug, args.threads)
  File "run_tracker.py", line 65, in run_tracker
    run_dataset(dataset, trackers, debug, threads)
  File "../pytracking/evaluation/running.py", line 54, in run_dataset
    run_sequence(seq, tracker_info, debug=debug)
  File "../pytracking/evaluation/running.py", line 21, in run_sequence
    tracked_bb, exec_times = tracker.run(seq, debug=debug)
  File "../pytracking/evaluation/tracker.py", line 58, in run
    output_bb, execution_times = tracker.track_sequence(seq)
  File "../pytracking/tracker/base/basetracker.py", line 35, in track_sequence
    self.initialize(image, sequence.init_state)
  File "../pytracking/tracker/segm/segm.py", line 163, in initialize
    self.init_segmentation(image, state, init_mask=init_mask)
  File "../pytracking/tracker/segm/segm.py", line 887, in init_segmentation
    self.params.masks_save_path, self.sequence_name, self.frame_name)
AttributeError:  'Segm' object has no attribute 'sequence_name'

How can I fix this mistake?
Thank you very much