bad performance(compare with cuMemcpy) on x86 system about gdrcopy HOT 3 CLOSED

GDRCopy uses CPU to push/pull data from GPU. It has lower latency compared with using GPU's copy engine, which is used by cuMemcpy (with some exception). However, CPU is generally not as good as the copy engine for moving large data.

To clarify, GDRCopy is not a replacement of cuMemcpy. The most efficient way is to use GDRCopy for moving small data and cuMemcpy for large data. You will need to find the cross point of these two on each platform you want to run your applications so that you know what sizes you should use GDRCopy.

from gdrcopy.

Hi @nkflash,

GDRCopy is designed for small message sizes. Based on what you have shown, GDRCopy is faster than cuMemcpy in that regime. Can you elaborate on the issue?

Also, you may want to run the application with numactl -C <cpus> -l or numactl -N <nodes> -l. It will perform better if you pin it to the CPU core that is close to the GPU in the PCIe tree. The -l will make sure that the host buffer allocation is from that numa node or CPU core.

from gdrcopy.

Hi @nkflash,

GDRCopy is designed for small message sizes. Based on what you have shown, GDRCopy is faster than cuMemcpy in that regime. Can you elaborate on the issue?

Also, you may want to run the application with numactl -C <cpus> -l or numactl -N <nodes> -l. It will perform better if you pin it to the CPU core that is close to the GPU in the PCIe tree. The -l will make sure that the host buffer allocation is from that numa node or CPU core.

Thank you for your quick response.
Sure, I see the small size is better than large size. In this case I didn't bind special CPU. Is that(bind to special cpu) comparable to 'cuMemcpy' in large size?

BTW: Why gdrcopy only suitable for small message size?

from gdrcopy.