How much memory does this cost? about manydepth HOT 4 CLOSED

@JamieWatson683 – as far as I remember we could train this model with a batch size of 12 on a single 12GB GPU, is that right?

Manydepth is pretty lightweight. Normal deep MVS methods tend to use a 4D cost volume, of size num_features x depth x height x width. We just use a 3D cost volume, of size depth x height x width. This helps a lot in terms of memory.

from manydepth.

Yes that sounds about right @mdfirman, maybe batch size of 10 for 12GB GPU for 640x192 (the "MR" resolution).

We generate a 3D cost volume, as we take the L1 distance between features at each location (collapsing the num_features dimension, rather than concatenating as per more heavy weight MVS methods.

We also don't let gradients flow through the cost volume or the feature extraction on previous images, which additionally reduces the memory consumption.

from manydepth.

Thanks for replying.

How do you process the cost volume to conver it to a single depth map? In my opinion, It's a 3D tensor so 3D conv is needed.

from manydepth.

Yes it's a 3D tensor – but we avoid 3D convs by treating the depth dimension as the channel dimension in standard 2D convs. So we treat the 3D cost volume as a 2D image with multiple channels. We then do a series of 2D convs to finally end up with a depth image.

from manydepth.