Query: Confusion about sudo requirement about gdrcopy HOT 3 CLOSED

Hi @tylerjereddy,

GDRCopy composes of 1) user-space library, 2) benchmark and test applications, and 3) driver. You need sudo to install the driver. The rest can be installed by normal users into folders that they have write permission. One scenario that you may want to install them separately is when using containers. Inside your container, you will want just 1) and/or 2). The driver should be installed on the baremetal. Then, you pass /dev/gdrdrv to your container when you launch it.

I am not sure about spack. If it installs only 1) and/or 2), you may have to install the gdrdrv driver separately. The gdrcopy package on spack seems to be required by ucx and nvshmem. Those libraries have a way to detect that GDRCopy is working properly on your system. If it does not work, ucx and nvshmem will silently switch to different algorithms. So, you probably do not see failure even if GDRCopy is not properly installed, unless you use GDRCopy directly.

from gdrcopy.

Hi Pak,

The context is this GROMACS issue: https://gitlab.com/gromacs/gromacs/-/issues/4846. Tyler is trying to use GROMACS with cuFFTMp (and hence NVSHMEM) on an HPC cluster, but is seeing:

WARN: GDRCopy open call failed, falling back to not using GDRCopy 
src/topo/topo.cpp:68: [GPU 7] Peer GPU 0 is not accessible, exiting ...

I'm not 100% sure that the GDRCopy warning and Peer GPU error are related, but a search of our internal Slack suggests they may well be so I suggested that he ensured that GDRCopy was properly installed on the system and tried again. Please let us know if you have further insight on this - thanks.

Alan Gray (NVIDIA Devtech)

from gdrcopy.

GDRCopy hasn't been a problem for my config recently, so closing.

from gdrcopy.