Task parallelism and warp-specialization about fuser HOT 5 OPEN

call for review: @naoyam @csarofeen @mmigdal-nv @drzejan2

from fuser.

The multi-device runtime by @samnordmann seems quite relevant. He's extending the overall design. Maybe we should also consider finer-grained task parallelism like warp specialization.

from fuser.

In case of matmul and use of TensorCores, the horizontal group with tasks will effectively mean partitioning of input tensors into sub-regions and within each of such tasks will be a vertical group with two tasks:

• handling data loading (TMA)
• process data with TensorCore
  ?

Or this is too low level, and for now by task we mean a higher level operation, not low level memory managment?

from fuser.

The multi-device runtime by @samnordmann seems quite relevant. He's extending the overall design. Maybe we should also consider finer-grained task parallelism like warp specialization.

I would be happy to discuss about it. I'll put here some design docs when it will be ready

from fuser.

In case of matmul and use of TensorCores, the horizontal group with tasks will effectively mean partitioning of input tensors into sub-regions and within each of such tasks will be a vertical group with two tasks:

• handling data loading (TMA)
• process data with TensorCore
  ?

Or this is too low level, and for now by task we mean a higher level operation, not low level memory managment?

You are right, it is to parallelize TMA and tensor core.

from fuser.