Why D2H is relatively slower？ about gdrcopy HOT 2 CLOSED

That text is quite old and is incorrect for coherent platforms, e.g. GH200 and POWER9+V100, where the GPU memory can be cached by the CPU. In that case, entire cache lines are exchanged and good performance is obtained.

Why GPU BAR cannot be prefetched?

On non-coherent platforms, the CPU can create MMIO mappings of the GPU BAR.
Prefetching per se refers to cache coherent platforms, and it does not apply to MMIO (UC) mappings.
Current CPU ISAs do not support reading or writing larger than 8B/16B granules when targeting MMIO mappings, even when using vector extensions.
On x86 platforms, Write-Combining (WC) mappings can improve write bandwidth, but do not help read performance.

Note that here I refer to CPU loads and stores, i.e. reads and writes generated by CPU instructions, which are interesting because of the their low latency properties.
Besides that, there can be DMA engines, similar to the GPU Copy Engines, which provide full performance in exchange of higher latencies.

In the README, no prefetched =>no PCIe burst reads transactions. Why the former leads to the latter?

That is an experimental statement.
When using cached mappings, loads and stores would trigger read/write transactions of entire cache lines, leading to large size PCIe transactions.
As of today, PCIe does not support a cache coherency protocol, so on non-coherent platforms, the CPU mappings of the GPU BAR cannot be cached (C). The best you can do there is to use MMIO or WC mappings.

I hope that helps.

from gdrcopy.

You have already solved my problem. Thanks a lot for your response!!!

from gdrcopy.