How to prevent holes during training? about monodepth2 HOT 1 CLOSED

Hi @dsGonz ,

Thanks for your interest in this. Where you have low texture regions, this breaks a fundamental assumption of self-supervised training (i.e. that depths can be recovered by matching pixels between camera views). Finding a method to fix this problem is a research problem in itself, and if you have interest and opportunity the community would be interested in efforts you make in this direction. Certainly some of the things you have suggested might work, or you could somehow detect low-texture regions and downweight their influence in the loss.

I agree that going for frame ids -1 0 +1 for a 30fpd sequence might be too small a baseline in general. You could consider using frame ids more widely spaced, e.g. -2 0 +2 or perhaps even more.

from monodepth2.