What GPUs have you used for training the Monodepth2 about monodepth2 HOT 6 CLOSED

@daniyar-niantic @mdfirman Finally it works when I turn to cudatoolkit 10 (installed by conda) and also change the cudnn version and pytorch version correspondingly : )

My experiment environment for RTX2080 Ti is:

Package Version
cudnn: 7.6.0
cudatoolkit: 10.0.130
pytorch: 1.1.0 (Build: py3.6_cuda10.0.130_cudnn7.5.1_0)
tensorboardx: 1.8

from monodepth2.

Hi @wuyujack

I suspect that the issue is due to CUDA and cuDNN versions and possibly pytorch version. Could you check the versions that you are using for RTX 2080 Ti and V100?

Thanks!

from monodepth2.

Agreed, looks related to this: pytorch/pytorch#12977

It would be useful for know if you are able to run any pytorch code on the RTX? What about a simple program which uses the offending torch.matmul? e.g.

import torch
a = torch.rand(3, 3)
b = torch.rand(3, 3)
torch.matmul(a, b)

If either of these fail then I think the problem is outside of our scope.

from monodepth2.

Hi @wuyujack

I suspect that the issue is due to CUDA and cuDNN versions and possibly pytorch version. Could you check the versions that you are using for RTX 2080 Ti and V100?

Thanks!

Thank you for your quick reply. For RTX 2080 Ti and V100 I both used cudatoolkit 9.0 (installed by conda) and py3.6_cuda9.0.176_cudnn7.4.1_1 and cudnn 7.6.0 (also installed by conda). Now I am trying to match all of them and also try the cudatoolkit 10 for luck : )

from monodepth2.

Agreed, looks related to this: pytorch/pytorch#12977

It would be useful for know if you are able to run any pytorch code on the RTX? What about a simple program which uses the offending torch.matmul? e.g.

import torch
a = torch.rand(3, 3)
b = torch.rand(3, 3)
torch.matmul(a, b)

If either of these fail then I think the problem is outside of our scope.

Also thank you for your quick reply! I will try and let you know in case that others can save their time to avoid using the RTX 2080 Ti if it doesn't work : )

from monodepth2.

Super, thank you! I will close this issue now.

from monodepth2.