twizwei / bddoia_project Goto Github PK

Hi @Twizwei ,

I am getting same predictions for all the examples in an epoch. And getting True only for Forward action even almost after 30 epochs now. And all false predictions for explanations.

Please see the below output for one epoch - I have printed the prediction tensors for actions and explanations. The prediction tensors are changing after many epochs, but for initial many epochs they were almost same for all examples. Can you please help with understanding what's getting wrong here..?

Output sample -

1. **********PRed tensor([[ 0.1388, -0.1472, -0.2860, -0.2082],
   [ 0.1387, -0.1459, -0.2868, -0.2056]], device='cuda:0')

**********Pred_reason tensor([[-0.6794, -1.6445, -1.2079, -1.2154, -2.5862, -4.5843, -4.7070, -1.1952,
-3.7522, -5.1099, -3.4112, -4.1208, -5.1506, -3.3282, -4.0545, -1.3703,
-1.4235, -1.6865, -1.0422, -1.5427, -2.2461],
[-0.6763, -1.6414, -1.2114, -1.2045, -2.5592, -4.4499, -4.5621, -1.1900,
-3.6692, -4.9239, -3.3501, -4.0153, -4.9572, -3.2591, -3.9511, -1.3657,
-1.4096, -1.6717, -1.0343, -1.5301, -2.2216]], device='cuda:0')

2. **********PRed tensor([[ 0.1390, -0.1459, -0.2867, -0.2049]], device='cuda:0')

**********Pred_reason tensor([[-0.6729, -1.6360, -1.2113, -1.2030, -2.5524, -4.4057, -4.5216, -1.1886,
-3.6488, -4.8690, -3.3328, -3.9836, -4.8995, -3.2408, -3.9205, -1.3616,
-1.4094, -1.6679, -1.0320, -1.5294, -2.2124]], device='cuda:0')

prediction action:
tensor([[ True, False, False, False]], device='cuda:0')
ground truth:
[[1. 0. 1. 0.]]
Accumulated Overall Action acc: 0.41973451327433625
prediction reason:
tensor([[False, False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False, False,
False]], device='cuda:0')
ground truth:
[[0. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0.]]
Accumulated Overall Reason acc: 0.0
(2259, 4)
(2259, 4)
Action Random guess acc:[0.52938689 0.47789276 0.35481963 0.37069922]
Action Random guess overall acc:0.3894205646860514
Action Category Acc:[0.70655597 0. 0. 0. ]
Action Average Acc:0.1766389922702548
Action Overall acc:0.41973451327433625
Action f1 macro Acc:[0.35327798 0.35567598 0.42824601 0.41461519]
Action mean f1 macro Acc:0.3879537918791195
Action f1 micro Acc:[0.54625941 0.55201417 0.74900398 0.708278 ]
Action mean f1 micro Acc:0.6388888888888888
Reason Random guess acc:[0.39808917 0.24130879 0.2985258 0.31220096 0.10874704 0.02166667
0.01204819 0.29882207 0.03355705 0.02903501 0.05110733 0.02449694
0.01728608 0.07351713 0.0312229 0.26969124 0.27478937 0.25506073
0.35023041 0.28089129 0.13421829]
Reason Category Acc:[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
Reason Average Acc:0.0
Reason Overall Acc:0.0
Reason f1 macro Acc:[0.4 0.45540019 0.44153276 0.43369265 0.48377514 0.49755338
0.49855716 0.43595506 0.49587146 0.49766511 0.4925876 0.49688196
0.49822301 0.49087221 0.49710597 0.44359606 0.44632353 0.45670996
0.42431193 0.45183208 0.47744622]
Reason mean f1 macro Acc:0.46742349636736286
Reason f1 micro Acc:[0.66666667 0.83621071 0.79061532 0.76582559 0.93714033 0.99026118
0.99424524 0.77290837 0.98362107 0.99070385 0.97078353 0.98760514
0.99291722 0.96414343 0.98849048 0.79725542 0.8061089 0.84063745
0.73705179 0.82425852 0.91367862]
Reason mean f1 micro Acc:0.8833870865743373

Thank you so much for your time. :)

Regards,
Vaishnavi Khindkar

Hi all,

Access was denied when I tried to download your dataset using the link below.

https://drive.google.com/open?id=1NzF-UKaakHRNcyghtaWDmc-Vpem7lyQ6

Could you please check this issue?

Thanks,

Hongbo

Hi @Twizwei ,

Thanks for sharing the repo.

I have been trying to run this on my local machine but was facing batch size issue so I used gradient accumulation with accumulation_steps as 4 and batch_size = 2. But I see training loss isn't quite decreasing.

I have attached the screenshot for training for 35 epochs. Can you please help with this , like is anything going wrong here?

Hi @Twizwei ,

Thanks for sharing this repo :)

I have downloaded lastframe.zip and BDD-OIA.zip as told for data. But I am not finding image data for val and test in there.

As stated in paper, model is trained on 16k images , I found that train image data in lastframe.zip but I can't find 2,270 val images and test set of 4,572 images in those folders. Can you please let me know where could I download them from for evaluation further?

Thanks and regards,
Vaishnavi Khindkar

Hi @Twizwei ,

Freezeing FasterRCNN option is set to false in code and also it's not set to true via arguments. But in paper it's mentioned that faster rcnn is freezed while training. I have changed it's default value to True as of now and have started training again.

Do I need to make any other change to replicate the results?

Thanks,
Vaishnavi Khindkar

Hi! @Twizwei ,When I train the model, run

python ./maskrcnn/maskrcnn-benchmark/action_prediction/train.py --batch_size 2 --num_epoch 50 --initLR 0.001 --gtroot "root-to-action-gt" --reasonroot "root-to-explanation-gt" MODEL.SIDE True MODEL.ROI_HEADS.SCORE_THRESH 0.4 MODEL.PREDICTOR_NUM 1 OUTPUT_DIR "output-directory" MODEL.META_ARCHITECTURE "Baseline1"

I got an error " AssertionError: Non-existent key: MODEL.SIDE". I am sure my environment configuration is ok.

Can you give me some advice？ Thank you!

Hey, in my opinion, the actions could be classified into 5 classes including forward, stop, left, right and confuse.

The corresponding one-hot code should be:

[1,0,0,0,0], [0,1,0,0,0], [0,0,1,0,0], [0,0,0,1,0], [0,0,0,0,1]

BUT when I check the validation data, I found a data labeled as [0,0,0,0,0].

What is this?

Attached pls find the screenshot. Thanks!

Hello!

In the paper there is no specified batch size in the text that was used for experiments. Could you clarify, was batch_size = 2 used for the experimental results, that can be seen in the paper?

Thank you!

it cannot be unzipped. Can you upload this clip to this repo please, the dataset BDD-100k is too large. THX