tud-ccc / cinnamon Goto Github PK

Update to latest when llvm/llvm-project#107005 is merged

Structure of the system:
Although the simulator is configurable, we will stick to Samsung's hbmpim paper parameters. With that said, in a single call to the device, a maximum of 16 banks, 64 channels, 1 rank, and 8 GRF can be configured.

cnm.workgroup { cnm.physical_dims = ["banks", "channels", "ranks", "grf"] } : !cnm.workgroup<16x64x1x8>

At the moment, the simulator implements different APIs for writing the data to the device for different operations. For example, preloadNoReplacement and preloadGemv function calls are used for elementWiseOperations and GEMV kernels. Because of this, the cnm.scatter is not directly lowered in the conversion pass; instead, depending on the operation in the launch block, the respective function is created.
Also, for all of this to work, we should limit the launch block to only a single operation.

We wrote that in the artifact description but it's annoying that the actual sources are in a cinnamon subdir. We should move the contents of that directory to the root.

Currently the UPMEM->DPU CPP pass generates one C file with just one main function.
We need to generate one binary for each upmem.launch though. The thread count is launch-specific, needs to be passed as compiler option when compiling DPU binary.

Draft design:

• Outline all DPU kernels into single module
• Have upmem->DPU CPP generate one function for each upmem.launch
• Have a main that conditionally compiles exactly one of those functions
• The conditional compilation variables along with the thread count are written in a header comment on line 1.
• This comment is parsed by a script that then invokes the DPU compiler and generates as many binaries as there are upmem.launch blocks.
• The binary path for each launch must be passed as argument to upmemrt_dpu_alloc.

• All CINM ops must be lowerable to Linalg/affine/whatever as a fallback if we don't use CNM
• We need a heuristic to determine when we should lower CINM to CNM vs when we lower CINM to CPU code

High priority

• #4
• Linkage issue with memrefCopy
• Checking that VA runs (@h4midf)
• Add histogram op (@h4midf)
  • In CINM
  • In CINM->CNM
• #7
• #6
• Write a README
• Remove artifact-specific stuff

Cost model (deadline 04.08)

Currently we have cinm.compute with attributes for workgroup shape and DPU memory size.
We assume this specification is correct, that is, the lowering pass cannot change them.
They should be obtained through the cost model.

• Implement a simple Samsung dialect
• Implement a pass that annotates Samsung and Upmem kernels with their time estimation
  • Implement the upmem cost estimator in C++

Lower priority

• Add verifier for shape of scatter map in UPMEM
• Fix the GPU lowering, was probably broken by recent changes to CNM

Optimization

• Hoist buffer alloc and free outside of loops
• Malloc avoidance
  • Avoid tensor reshapes that do a copy (Especially for VA that's a problem)
  • Unify buffers across loop iterations
• Affine map simplification with dimension sizes

This is in part enforced by the CNM scatter map shape. The scatter map mandates that the last dimensions of the input have the same shape as the buffer. However since then, CNM scatter has been made bufferizable. If the memref has a custom layout or strides, the elements may not be contiguous anyway.

For instance

%3 = scf.for %arg6 = %c0 to %c1024 step %c32 iter_args(%arg7 = %alloc_0) -> (memref<1x128xi32>) {
          %subview_2 = memref.subview %arg0[%arg2, %arg6] [1, 32] [1, 1] : memref<1x1024xi32> to memref<1x32xi32, strided<[1024, 1], offset: ?>>
          %4 = upmem.alloc_dpus : !upmem.hierarchy<1x128x1>
          upmem.scatter %subview_2[132, 32, #map] onto %4 : memref<1x32xi32, strided<[1024, 1], offset: ?>> onto !upmem.hierarchy<1x128x1>

is this legal?
-> yes, because the stride in the second dimension is 1.