OpenCL error with `CL_MISALIGNED_SUB_BUFFER_OFFSET` about halide HOT 2 OPEN

Ugh, if some devices require that alignment, I think we need to reconsider using sub buffers at all, because we can't guarantee that sort of crop alignment.

from halide.

This happens when passing cropped device buffers to an OpenCL kernel compiled by Halide. Halide implements halide_device_crop for OpenCL by storing an offset in the Halide buffer structure. In order to pass a buffer with a non-zero offset to an OpenCL kernel, at least as Halide currently generates code for OpenCL, a sub-buffer must be created or the crop area copied to a new device buffer. The halide_opencl_run calls clCreateSubBuffer to create a sub-buffer as this is more efficient than copying. (And since the buffers are general read/write, copying requires copy-in/copy-out, though it is possible we have enough information to eliminate one side or the other in many cases.)

Unfortunately, sub-buffers must be aligned to the CL_DEVICE_MEM_BASE_ADDR_ALIGN value, which cannot be assumed to be 1. On devices where stricter alignment is required, a cropped device buffer can potentially cause a failure in halide_opencl_run as documented here. The issue does not include a reproducing case, but the easiest way to make a test case is to call halide_device_crop to form a buffer with an offset of 1 and pass that to Halide generated computation scheduled using OpenCL.

There are three possible ways to go here:

1. Declare it an error. Likely improve the reporting in halide_opencl_run and document how it occurs.

2. Query the OpenCL device for the required alignment and if the supplied input or output buffer does not meet the requirements, have the runtime copy it to a newly allocated buffer. This involves taking care to copy in and copy out as required. The copies can likely be asynchronous but the allocation and free probably have to serialize somewhat. And as with many things where we take care of this inside the runtime, there is little opportunity to reuse the temporary buffer or otherwise optimize/amortize the cost. That said, the cost would only be incurred when the current implementation would crash. We should likely think a bit on whether copying preserves semantics in all cases. E.g. aliasing.

3. Compile the OpenCL kernel code to take an offset in addition to the buffer. Doing this for every single buffer is likely enough of a performance issue to merit programmer control in the schedule. I.e. one would have to mark something as unaligned to support this. (Or optionally, introduce a concept akin to host alignment in the schedule for GPU to allow generating more efficient code when proper alignment is asserted. However this is fairly dubious as the required alignment is not known at compile time, only runtime.)

Option 2 is likely the way to go. It's not a one liner, but it's not that difficult to code. It's more tricky to test without a platform handy that exhibits the behavior.

from halide.