InfiniBand support about sarus HOT 1 OPEN

Hi @mvhulten, at the moment we have no plans for a dedicated InfiniBand hook.
I have limited experience with InfiniBand, but it should be possible to make IB-based networks work with the current MPI hook.
Taking as example an MPI hook configuration we have for a system at CSCS:

• ibverbs libraries (e.g. /usr/lib64/libibverbs.so.1 and /usr/lib64/libibverbs/libmlx5-rdmav25.so) should be added to MPI_DEPENDENCY_LIBS;
• the path for drivers config files (e.g. /etc/libibverbs.d) should be added to BIND_MOUNTS;
• InfiniBand devices (e.g. /dev/infiniband/issm0, /dev/infiniband/rdma_cm, /dev/infiniband/umad0, and /dev/infiniband/uverbs0) should be added to BIND_MOUNTS

In case you would like the IB network to be exposed into any container, an alternative option could be to define ibverbs libraries and config directory as siteMounts and the IB devices as siteDevices.

I hope these tips are useful!

from sarus.