bertinetto / cfnet Goto Github PK

From my understanding of the code, you dont model the width and height of the target object. Is there any specific reason for this? or did i understand the code wrong?
thanks in advance

I found that the Gaussian response in the gaussian_response.m is not the 2D Gaussian shaped peak centred on the centre of the image. I plot the gaussian produced by the code:

Why the peak is at the corners? Is that on purpose or a bug?

I suppose the gaussian response should look like this:

I found the difference is at:

If change the code to:

function [i1, i2] = circ_grid(m1, m2)
    m = [m1, m2];
    [i1, i2] = ndgrid(0:m(1)-1, 0:m(2)-1);
    half = floor((m-1) / 2);
    % i1 = mod(i1 + half(1), m(1)) - half(1);
    % i2 = mod(i2 + half(2), m(2)) - half(2);
    i1 = i1 - half(1);
    i2 = i2 - half(2);
end

The peak of gaussian response will the centre of the image.

Hi @bertinetto
I editing the files like





and I run the run run_cfnet2_evaluation.m
get the result

please tell me where have problems

Hi Luca, thanks for your great work! I have a question about scale changes. I have some sequences with high scale variations. I have tried some popular trackers (DSST, ECO, ATOM etc.) by arranging hyperparameters and I got good results. But when I tried to run CFNet or SiamFC somehow I could not get good results. So I want to you ask that are there only these 4 hyperparameters for scale changes?

1. scaleStep
2. scalePenalty
3. scaleLR
4. numScale

Are there any other parameters should I arrange? Also when I decrease scalePenalty, I expected that bbox can change easier but I observed that it doesn't. Should I increase the penalty to give flexibility to the bbox?

Thank you!

I had a problem when I run the 'run_OTBReadyTracker(seq)'.

Undefined function or variable 'MulConst'.

Error in dagnn.DagNN.loadobj (line 26)
block = constr() ;

Error in tracker (line 83)
net_z = dagnn.DagNN.loadobj(net_z.net);

Error in run_OTBReadyTracker (line 25)
[bboxes, ~] = tracker(tracker_params);

Error in my_run_traxker (line 9)
results = run_OTBReadyTracker(seq);

• function [branch1, branch2] = make_branches_alexnet(opts)
  Above function creates a branch_opts struct with batchNormalization set to true

ie, branch_opts.batchNormalization = true;

then passes branch_opts to make_branch_alexnet

f = @() make_branch_alexnet(branch_opts);

branch1 = f();

BUT make_branch_alexnet_nodilation is called because it is first function in make_branch_alexnet.m , if you don't understand what i mean here go : https://www.mathworks.com/matlabcentral/answers/396964-why-the-function-file-should-have-the-same-name-as-the-first-function-in-the-file

However, branch_opts (the passed argument) is not used in the make_branch_alexnet instead a newer version is created.

function net = make_branch_alexnet_nodilation(varargin)
opts.exemplarSize = [127 127];
opts.instanceSize = [255 255];
opts.last_layer = 'conv5';
opts.num_out = [96, 256, 384, 384, 256];
opts.num_in = [ 3, 48, 256, 192, 192];
opts.conv_stride = [ 2, 1, 1, 1, 1];
opts.pool_stride = [ 2, 2];
opts.scale = 1 ;
opts.initBias = 0.1 ;
opts.weightDecay = 1 ;
opts.weightInitMethod = 'gaussian';
opts.batchNormalization = false ;
opts.cudnnWorkspaceLimit = 102410241024 ; % 1GB
opts = vl_argparse(opts, varargin) ;

BUT batchNormalization does not match!!!!
Why is the purpose behind creating the a new object instead of

opts = varargin{1};

Thank you, this is an excellent work.

hi Luca,

So how to set random vectors of hyperparameters?

thanks

After reading the code, I realized that the aspect ratio is fixed throughout the tracking, because the make_scale_pyramid function creates the pyramid evenly along with the width and height.

So the question is: how to adapt to changing aspect ratio?

Thanks!

Hi Luca,
You got every thing well prepared for reproducing the experiment:+1:.
Meanwhile, due to many factors, errors may occur, I got

Error in dagnn.DagNN.loadobj (line 26)
    block = constr() ;

Error in tracker (line 84)
    net_z = dagnn.DagNN.loadobj(net_z.net);

Error in run_tracker_evaluation>do_OTB_TRE (line 107)
        [new_boxes, new_speed] = tracker(tpar);

Error in run_tracker_evaluation (line 63)
            [all_boxes, all_gt, all_type, times] = do_OTB_TRE(video, all_boxes, all_gt, all_type, times, run_params.subSeq,
            tracker_params, run_params);

and assume this issue correlates with matconvnet version and hope more detailed about used matconvnet in original experiment could be provided, many thx:smiley:!

Hi @bertinetto ,
thanks for open sourcing this brilliant work!
I want to run demo/evaluation.
After downloading pretrained nets and validation videos.

I executed run_cfnet1_evaluation.m but I have the following error which I could not solve

Invalid MEX-file 'C:\matconvnet\matlab\mex\vl_imreadjpeg.mexw64': The specified module could
not be found.

Error in load_video_info_votformat (line 37)
    imgs = vl_imreadjpeg(img_files,'numThreads', 12);

Error in run_tracker_evaluation>do_OTB_TRE (line 74)
    [tpar.imgFiles, ground_truth] =
    load_video_info_votformat(fullfile(tpar.paths.eval_set_base,rpar.dataset), video);

Error in run_tracker_evaluation (line 51)
            [all_boxes, all_gt, all_type, times] = do_OTB_TRE(video, all_boxes, all_gt,
            all_type, times, run_params.subSeq, tracker_params, run_params);

Error in run_cfnet1_evaluation (line 12)
[~,~,dist,overlap,~,~,~,~] = run_tracker_evaluation('tc_Kite_ce3', tracker_par);

Could you give me any suggestion how can I solve this problem please?

I am calling vl_imreadjpeg() without an eror inside matconvent though

I also check the dependencies of vl_imreadjpeg.mexw64, however the output log looks scary and I have no idea how to interpret it. I mean there are too many dependencies if I were to install them one by one. I think there should be more shorter way.

I have:
CUDA v8.0
CUDNN v5.1
Matlab 2016a

Thx

hi, I run the Tracking only code,bug I meet a bug:No property dilate for a layer of type dagnn.Conv, the version of matconvmat is v1.0-beta20. Can you help me?

Hi, is there any plans for releasing a Tensorflow or Pytorch based version of this codebase?

thanks,
Aruni

Hi Luca. Thanks for your work!

I would like to fine tune your pre-trained Imagenet weights using my smallish video data.

The object I need to track is not one of the classes in Imagenet. I used this implementation out of the box, it works fairly well, but I would like to improve the accuracy by training/tuning on my own data. I was wondering approximately how many new videos and frame pairs would I need to get good results. ?

Hi Bertinetto,

I am trying to run your code but I don't have an GPU.
Is there anyway that I can modify the code?
I saw there are several pieces related to GPU and I am hesitating to make major changes.

Best Regards,
Cong

I follow the tutorial configuration, but I can't find paths.stats = 'J:/cfnet-master/data/ILSVRC2015.stats.mat'
in env_paths_tracking.m，and when I run the run__evaluation,the Matlab reported error:Wrong usage load.Unable to read file 'J:/cfnet-master/data/ILSVRC2015.stats.mat' ,no such file or directory.

Hi,

good work as always : )

The speed of CFNet-conv2 reported in the paper is 75 fps. But when I evaluate this model in OTB-2013, it ran less than 3 fps. I am using NVIDIA GTX 1080, cuda 8.0, cudnn 5.1. Matconvnet is compiled with GPU.

Is there anything I might be missing?

Hi, Luca and Jack.
good work as always.
I run the CFNet-conv2, CFNet-conv5, Baseline-conv5 on OTB2013 OPE by setting the hyperparameters according to the table2 in the paper and find that the results are much worser(the numbers after the "/" are the results shown in the paper):

1. Baseline-conv5:
precision:0.779 / 0.806
success:0.6 / 0.618

2. CFNet-conv2:
precision:0.737 / 0.807
success:0.561 / 0.611

3. CFNet-conv5:
precision:0.723 / 0.803
success:0.518 / 0.611

Does anyone knows what may cause these results? I run the code on Matlab2016a by using CUDA8.0 and matconvnet-25

Hi, Luca.
Thanks for good work.
To train network for gray and color separately as the networks given, how to set parameters(such as grayscale) for gray and color?
Thank you very much.

I followed "Tracking only" 1~6 step.
and Matlab gave me this message "dist: 32.25 overlap: 60.94 fps: 32.0" when I ran 'run_cfnet1_evaluation' with 'tc_Airport_ce' dataset.
I expected the result show me 'tracking scene' because this model is suited to object tracking, but just threw me above message.
Originally, does this model throw 'distribution result, overlap (IoU), fps (frame per second) ??

Great work ,guys!
can you tell me the exact version of matlab and matconvnet you guys use in this project?
because i met some problems with loading the pre-trained nets
and it says:
A dot name structure assignment is illegal when the structure is empty. Use a subscript on the
structure. when i try to call the dagnn.DagNN.loadobj() in tracker.m

Hi bertinetto~I run cfnet-conv5_e80 network how overlap only 37.07? Is it because of the parameters?

Hello, I am currently trying implementing your network but when I run the tracker I got the following message.

||| OTB-TRE evaluation for 0 sequences and 3 sub-seq each |||
dist: NaN overlap: NaN fps: NaN

Could you please tell me what is going on?
Thanks

hi,luca!
What kind of grayscale image do you use to re-train each architecture with grayscale images?

Hi bertinetto:
Thanks for your excellent work. I have successfully run this tracking program. But I ran training program with trouble. The vl_imreadjpeg could not read image. The warnning information is like this: 'E:\code\tracking2018\cfnet0502\data\d\ILSVRC2015_train_01117000\000004.00.crop.x.jpg' because 'gdi+: InvalidParameter (orimage file does not exist)' .So could you tell me the problem?Or give me some hint?

Hi, thank you for your attention. An error occurred when I compile vl_complienn only though cpu. My system is intel i3-7350K, 1050ti, ubuntu16.04, Matlab 2016b, Gcc 4.9, cuda 9.0

Error using mex
/home/river/NewFolder/toolbox/matconvnet-1.0-beta25/matlab/mex/.build/vl_imreadjpeg.o：In function
‘Batch::Item::Item(Batch const&)’：
vl_imreadjpeg.cpp:(.text+0x6ef)：undefined reference to ‘__warn_memset_zero_len’
collect2: error: ld returned 1 exit status

error in vl_compilenn>mex_link (line 627)
mex(args{:}) ;

error in vl_compilenn (line 500)
mex_link(opts, objs, flags.mex_dir, flags) ;

Any suggestions will be appreciated. Thanks in advance.

Hi Luca,

Thanks for the amazing work and the code!

I have a question about the search region extracted every frame. The paper says that the search region has an area that is four times the size of the target object. Is there any particular reason that you choose "four times"? I was wondering whether there is an easy way to adjust the area search region. The file src/tracking/make_scale_pyramid.m seems to have related content but I wasn't able to make successfully changes.

Thank you for your help!

sorry , is there any files that i can use for the path of net_base and eval_set_base in env_paths_tracking.m ?

Hello~
the video sequence is too large for me to download it form google drive , can any one tell me the official website to download it ?

thank you

Su Zhengpeng

hi,luca!
in file cfnet/src/tracking/tracker_step.m line:42
% displacement from the center in instance final representation ...
disp_instanceFinal = p_corr - (p.scoreSize*p.responseUp + 1)/2;
% ... in instance input ...
disp_instanceInput = disp_instanceFinal * p.totalStride / p.responseUp;
% ... in instance original crop (in frame coordinates)
disp_instanceFrame = disp_instanceInput * s_x / p.instanceSize;
% position within frame in frame coordinates
newTargetPosition = targetPosition + disp_instanceFrame;

for disp_instanceFrame = disp_instanceInput * s_x / p.instanceSize
s_x is scale coefficient;I think it depend on the best_scale you find. not always s_x;

I prepared my training data in the format of VID, and generate the training data as the curation, like this:
imdb_video =

struct with fields:

             id: [1×168 uint32]
n_valid_objects: [1×168 uint32]
        nframes: [1×168 uint32]
        objects: {1×168 cell}
           path: {1×168 cell}

total_valid_objects: 83764
valid_per_trackid: {50×168 cell}
valid_trackids: [50×168 uint16]
set: 1
But error occured:
Undefined function or variable 'ideal_size'.

Error in make_branch_alexnet (line 76)
[ideal_exemplar, ~] = ideal_size(net, opts.exemplarSize);

Error in make_net>@()make_branch_alexnet(branch_opts) (line 48)
f = @() make_branch_alexnet(branch_opts);

Error in make_net>make_branches_alexnet (line 49)
branch1 = f();

Error in make_net (line 13)
[branch1, branch2] = make_branches_alexnet(opts);

Error in experiment (line 80)
net = make_net(opts);

Error in run_experiment_baseline_conv5 (line 19)
experiment(imdb_video, opts);

Can you help me ?

How do I run this tracker for my own data?

In CFNet's table 1, it shows that the baseline+CF-conv3 gives the best result. But how can I train this network by myself?
I only show the code in src/training/run_experiment*, which gives the manner to train CFNet-conv1 CFNet-conv2 CFNet-conv5 and baseline-conv5.
Thank you~

I get a problem when run run_cfnet1_evaluation.m, the main questioned code isnet_z = dagnn.DagNN.loadobj(net_z.net);. I use matconvnet-1.0-beta24, is the version of matconv wrong?

Hi,
First of all thank you very much for publishing the code.
In the 2nd conv layer in the pre-trained network "cfnet-conv2_gray_e40.mat" (and also in other pre-trained) the number of channels is 48 (5x5x48x32) while the input(to conv2 layer) num of channels is 96.
I thought the reason is "group" but i cant see the rest of the 5x5x48x32 filter, hence the result of conv2 layer take in account only the first 48 input channels.
Do this deliberately?
or accidentally, the 2nd group (the rest 5x5x48x32 filters) do not saved in the mat file?

Thank you

Hi,

Your work is really great!
I have some questions when I train the cfnet-conv2 network. In your code 'run_experiment_cfnet_conv2.m', the epoch is set to 100. But in your pretrained networks, the pretrained networks for cfnet-conv2 are called 'cfnet-conv2_e80.mat' and 'cfnet-conv2_gray_e40.mat'. I want to know how many epochs do you set when training the cfnet-conv2 network (color and gray).

Thank you very much!
Hope your reply!