question about the predictor machanism about triton HOT 5 CLOSED

Hi :)

The .json file contains the predictor (random forest) to pick the right kernel template to choose given the input shapes. The predictor is trained in $PROJECT_ROOT_DIR/tune/, and serialized into a .json file (to make sure that ISAAC doesn't depend on Python.).

What kind of kernel are you trying to add?

from triton.

Thanks for your reply. currently we have some fine-tuned kernel of GEMV/GEMM for Intel's graphics, the kernels have some issues:

1. they may not suitable for all the GEMV/GEMM parameters.
2. the other is that they may not have good performance for non-Intel platform.
   what is your opinion about the good solution that integrated these kernels into ISAAC?
   add @gongzg into discussion

from triton.

Oh, yes. I've talked briefly to @gongzg about that. For now I just know that the kernels used intel subgroups extension, which is fine. I have a few questions, so that I can think about integrating them:

Does it work for all sizes of M, N, K (i.e., handles bounds-checking properly) without calling additional "cleanup" kernels?
Does it work for the 4 layouts (NN, NT, TN, TT)?
What are the tunable parameters (how many?). In the current GEMM generator, one parameter ("Depth") is pretty useful for handling "small M, N ; large K" situations. And I'll add another one. We can talk about adding them into your kernel template, if they're not already there.

from triton.

Hi,
Currently our kernels support al the sizes of M, N, K, but will bring cleanup kernels.
And just work for RowMajor layout currently.
Don't have tunable parameters, but have some different kernels for different usage situation.

from triton.

That could be interesting :) Is there any place where I can see and/or benchmark the code?

from triton.