ibaer / tiny-dpcpp-nn Goto Github PK

This repository implements a GPU-accelerated tiny neural network framework using Intel hardware, based on the original CUDA implementation. The implementation uses the Intel DPC++ compiler and relies on the SYCL language with optional ESIMD acceleration.

The network is optimized for loading both activation matrices and weights matrices into the GPU's fast L1 memory and registers. Computation of matrix multiplications is executed using Intel's joint_matrix extension, a high-level wrapper for systolic array operations.

We benchmarked the thoughput of our network in training and inference on both Intel Data Center GPU Max Series (Ponte Vecchio) and Intel Arc Series and compared our network with PyTorch.

To replicate the performance of the dpcpp code, please set BUILD_BENCHMARK=ON in tiny-dpcpp-nn/CMakeLists.txt, build benchmark-all and run the benchmark from the build/ folder using

I_MPI_DEBUG=3 I_MPI_OFFLOAD=1 I_MPI_OFFLOAD_DOMAIN=[1,2] mpirun -n 2 ./benchmarks/benchmark-all

To replicate the performance of the pytorch code, please run

cd python/ && python benchmark_pytorch.py

Finally, plot the results using

python benchmarks/plot_results.py

We reach up 60x to compared to PyTorch:

We reach up to 20x compared to PyTorch:

• High-Performance Computing: Optimized to run efficiently on Intel Data Center GPUs, enabling high-throughput training and inference with up to 60x over PyTorch.
• Compatibility with PyTorch: Provides Python bindings that integrate seamlessly with the PyTorch ecosystem, enabling users to include GPU-accelerated MLPs in PyTorch applications.
• Versatile Neural Network Structures: Supports networks with multiple hidden layers and a variety of neuron configurations to fit different use cases and performance requirements.
• Multi-Resolution Hash Encoding: Includes implementation of Multi-Resolution Hash Encoding, allowing the network to handle high-frequency features effectively.
• Cross-Platform Utilization: Designed to be run on various Intel GPUs, maximizing the portability and usability of the framework across different systems.

For a detailed documentation, please refer to tiny-dpcpp-nn documentation.

To build the tiny-nn librairy, you can clone the github repo on your machine and put your code in the source folder. After cloning, if you choose to use the pybindings, please recursive pull the pybind11 repositories via

git submodule update --init -- extern/pybind11

If you also want to pull the reference unittest data in test/tiny_dpcpp_data, which are ~500 MB of reference inputs, outputs, and weights, you can also run git submodule update --init. Note, that if BUILD_REF_TEST=ON in CMakeLists.txt, then test/tiny_dpcpp_data will be cloned as well.

Then you can build the library using :

source /opt/intel/oneapi/setvars.sh
mkdir build && cd build/
cmake -D<options>=<ON/OFF> ..
make

where are options that can be toggled on or off. See Build Options

Note: To make the use of the network, you have to disable the implicit scaling on PVC which can be done by uncommenting the portion of the code indicated in the sample when creating the queue.

We provide a pybind wrapper of our tiny-dpcpp-nn implementation for seamless integration into PyTorch. Please refer to tiny-dpcpp-nn pybind documentatino

Install the latest ipex via

python -m pip install torch==2.1.0a0 torchvision==0.16.0a0 torchaudio==2.1.0a0 intel-extension-for-pytorch==2.1.10+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

When setting the additional flag BUILD_REF_TEST=ON, additional data from tiny-dpcpp-data will be downloaded.

When setting the additional flag BUILD_TORCH_TEST=ON, the libtorch tests (tnn_api.h) will be built.

To have all tests, run:

cmake -DTARGET_DEVICE="PVC" -DBUILD_REF_TEST="ON" -DBUILD_REF_TEST="ON" ..

After all tests are build, you can run bash test/run_tests.sh to verfiy that the setup is correct. Please note that we provide tests for both the core dpcpp implementation and the libtorch wrapper implementation.

To test whether the pytorch bindings were installed correctly, please run

• pytest python/tests/test_compare_torch_dpcpp.py to ensure that forward and backward passes work properly.
• python/tests/test_training_classification.py and python/tests/test_training_regression.py to see if integration into PyTorch's optimiser works, and

• The repository was developed and is maintained by Christoph Bauinger ([email protected]) and Kai Yuan ([email protected]).
• The original implementation of SwiftNet was conducted by Darius Dabert (DariusDabert) and Adrien Tousnakhoff (Tousnaaa)

If you found this work useful, please consider citing this work as:

@software{tiny-dpcpp-nn,
	author = {Bauinger, Christoph and Yuan, Kai},
	license = {BSD-3-Clause},
	month = {3},
	title = {{tiny-dpcpp-nn}},
	url = {https://github.com/intel/tiny-dpcpp-data/},
	version = {0.1},
	year = {2024}
}