Add/test broadcasting support to ROCArray about amdgpu.jl HOT 6 CLOSED

Yeah this decision is always a hard one. What to do with heterogeneous arrays or arrays on different devices. GPUArrays currently fails on heterogenous arrays.

For mixed devices, I presume HSA supports page-faulting memory in from a different device? If so I would choose a device (were that choice can be informed or a guess).

from amdgpu.jl.

Yes, we only allocate so-called finegrained memory, which is page-fault friendly. If you see my latest pull of memory.jl into the codebase, all upload/download/transfer operations are just implemented as ccall(:memcpy) since it's technically valid and I'm too lazy to figure out how to be efficient right now.

What I think we can/should do when there are multiple devices involved in OOP operations is just allocate solely on the default GPU (the one set by set_default_agent!), and then rely on contextual dispatch to change that value wherever the user wants to use a different default.

from amdgpu.jl.

Fat-fingered the close button, thanks phone!

from amdgpu.jl.

Oh, I see what you meant about page-faulting from another device. That would need the ROCm RDMA stuff working most likely. I do plan on implementing that eventually but not right now. Currently we'll just do cross-GPU memory transfer on the host.

from amdgpu.jl.

Transferring issue (instead of closing) in case this still needs to be addressed for ROCArray

from amdgpu.jl.

We now allocate coarse-grained memory for ROCArrays, we just allocate new arrays onto the default device, and we have an upload/download/transfer API, so I'm closing this.

from amdgpu.jl.