This is the OpenCL implementation of the CMS standard "three threshold" algorithm for FPGA. It is based on the parallel CPU and GPU version availble at https://github.com/beiwang2003/strip_clustering_gpu. The prefix sum implemenation is based on the clpp library at https://code.google.com/archive/p/clpp/downloads
-
setup the env for FPGA
source /opt/intel/fpga-d5005/inteldevstack/init_env.sh (this is the default 19.2 OpenCL SDK. We can load 19.4 and 20.1 OpenCL SDK using source /opt/intel/fpga-d5005/inteldevstack/init_env_19_4.sh and source /opt/intel/fpga-d5005/inteldevstack/init_env_20.1.sh) -
compile the *.cl kernels in emulation mode for debugging
aoc -march=emulator ./clppScan_Default.cl
aoc -march=emulator ./clustering.cl
aoc -march=emulator ./findSeedStrips.cl
(note: option -legacy-emulator is required for compiling using 19.2 OpenCL SDK. For newer version, e.g., 19.4 and 20.1, the compiler is using -fast-emulator option by default)
The above steps will result in 3 different directories, one for each file. To generate a single .cl file, we can use #include ".cl" to include the first two in the third one.
-
setup the env for the host module load rh/devtoolset/9
-
compile the host code by running Makefile
make
(use EMULATOR flag in the makfile for running in emulation mode) (use make -n to check the the compiler options added to the compiling process) -
run the code in the host (for 19.4+ version)
env LD_LIBRARY_PATH=/usr/licensed/anaconda3/2020.7/lib:$LD_LIBRARY_PATH ./strip-cluster
(use ldd ./strip-cluster to check if the object file is linked to the right shared libraries)
Note: the emulator in Prof Edition is built with GCC 7.2.0 and so the libstdc++.so have to be at least as new as provided in GCC 7.2.0, libstdc++.so.6.0.24. See: https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/hb/opencl-sdk/aocl_getting_started.pdf#page=39. The devtoolkit provided at RHEL 7 system at Princeton does not provide the required libstdc++ version. Fortunately, anaconda carries libstdc++.so.6.0.26 which is from GCC 9.1.0. To link to that library, we will need to set up the env as indidcated ablove
-
intermediate compilation with -c option (generate <kernel_filename>.aoco file)
aoc -c <kernel_filename>.cl -report -v -
intermediate compilation with -rtl option (generate <kernel_filename>.aocr file)
aoc -rtl <kernel_filename>.cl -report -v -
review html reprt at <kernel_filename>/reports/report.html
4*. simulate (generate <kernel_filename>.aocx file>
aoc -march=simulator <kernel_filename>.cl
5*. compile (generate <kernel_filename>.aocx: minutes to hours)
aoc -fast-compile <kernel_filename>.cl (fast compilation for small design changes)
aoc -incremental <kernel_filename>.cl (incremental compilation for big design changes)
- full deployment with profiling turned on (generate <kernel_filename>.aocx: hours)
aoc -profile <kernel_filename>.cl -report
-
setup the env for the host module load rh/devtoolset/9
-
compile the host code by running Makefile
make
(do NOT use EMULATOR flag in the makfile for running on board) -
run the code in the host (for 19.4+ version)
env LD_LIBRARY_PATH=/usr/licensed/anaconda3/2020.7/lib:$LD_LIBRARY_PATH ./strip-cluster
(this will generate a profile.mon file for compiling with -profile option for the *.cl file) -
Post-processing the profile.mon data into a readable profile.json file for data processing with Intel VTune aocl profile ./strip-cluster -x unifiedKernels.aocx -s unifiedKernels.source (this will generate a profile.json for further data processing with Intel VTune)
-
Instead of using Intel VTune, we can also launch the Intel FPGA Dynamic Profiler for OpenCL GUI aocl report *.aocx profile.mon *.source
-
query the device name of your fpga board
aocl diagnose (look for device_name, e.g., acl0, alc1, ...) -
run a board diagnostic test
aocl diagnose device_name -
program the FPGA offline or without a host
aocl program device_name <kernel_filename>.aocx
Note: it is important to program the board before running board diagnostic test, e.g., aocl diagnose device name
-
- compile the host code by running Makefile
make
- compile the host code by running Makefile
-
run the code in the host
./strip-cluster