Single stream rgb is working but double stream not about admd HOT 11 CLOSED

So, not really sure what the problem is.
Single rgb works fine
Single depth works fine
Double stream does not work, is that correct?

from admd.

After you train rgb and depth, you should have the trained checkpoints saved in the model folder.
When you train the double_stream, the two branches of the network are initialized with those checkpoints. From your error, it look like you do not have the depth checkpoint there.

from admd.

I think my path was wrong. Also when in training it's written test acc. do you really check the accuracy on the test set at every epoch?

from admd.

Well at the beginning I was just testing at the end with the --mode test_xxx options. Early stopping is done when loss is nearly stable since there is no validation set for this dataset. I inserted the test check during training just to verify everything was ok. But in no way test accuracy is used for validation.

from admd.

Ok it's working but I dont really undertstand I dont get performance like you there must be a bug, just to recap what I do for training the double stream, I first train RGB and depth single stream after I start the training for the double stream, at start the test. acc is 54% but after first epoch it decrease to 37% and at the end of the 2000 iterations its even lower, around 23% acc. I didnt change anything in your code.

from admd.

Hi, as we state in the paper, finetuning the two-stream does not help much and is often harmful.
Simply taking the two separate streams (trained independently) and average logits works.
For NYUD I think I only finetuned for 1000 iterations with a very low learning rate.
Please see

Do you get a reasonable accuracy for the two separate RGB and Depth stream (cf. Table 5 in the paper)?

from admd.

I'm gonna give you numbers tomorrow or after tomorrow, but it was less than what you put in your paper to be frank.

from admd.

This is what I get, which is the number in the paper.

from admd.

What is your training strategy ? What do I have to train: single stream RGB, single stream Depth, double stream after that (?) and finally Hallucination network ? I followed your paper, but certainly I was wrong somewhere.

from admd.

Yes, that is the idea. In the file run.sh you have everything, also some other baselines such as modDrop and rgb ensemble.

from admd.

Closing this since inactive for a week

from admd.