How do you evaluate the kitti eigen split? about adabins HOT 2 CLOSED

The paper uses Garg crop as defined here. All the crops are optional (technically) and you can easily enable or disable them during both training or testing by passing (or not passing) the respective arguments. See train.py and evaluate.py for the arguments explanation. Note that the Garg crop is commonly used as an evaluation mask and cuts off a significant portion (~40%, mostly in the sky-region) whereas kb_crop is more like an optional preprocessing step (e.g. Official dev-kit) for stripping off invalid regions (only about ~2-6%). You can refer to previous works such as this or this for usage.

For KITTI, official annotated depth data is used for both training and testing, found here: http://www.cvlibs.net/datasets/kitti/eval_depth.php?benchmark=depth_prediction

from adabins.

Hi @shariqfarooq123, Thanks for releasing the code of your work!

I've just ended with the same doubt as @xingyi-li. You pointed to previous work BTS where they do the kb_crop also for evaluating. I've just realized that BTS performs the crop on the input image, but then, when evaluating they use the full, uncropped ground truth. What they do is to fill the cropped predicted depth to match the ground truth depth.

On the other hand, on your code, when evaluating you perform first the kb_crop and then the garg_crop, what means that you are evaluating with a considerable smaller portion of the ground truth. Evaluating on the ground truth only with the garg_crop (and inputs with both crops as in BTS) worsens the results.

I am missing something? or is there any other work that performs both crops on the GT?,

thanks a lot!

from adabins.