Package Request: python-pytorch-all about arch4edu HOT 4 OPEN

Tensors and Dynamic neural networks in Python with strong GPU acceleration (with TensorRT, CUDA, ROCm, OneAPI-DNN (MKL-DNN), ZenDNN and AVX2 CPU optimizations)

Great idea but I'm not sure if it's possible to compile pytorch with both CUDA and ROCM enabled.

• With CUDA 9 or 11 all nVIDIA sm20, sm3x cards (Fermi, Kepler: Notable cards include the 6GB Quadro 6000, Quadro 7000, 2x 6GB Quadro Plex 7000; 2x 12GB Tesla K80, 12GB Tesla K40, the 6GB cards below that; also the 6GB Titan and Titan Black and 2x 6GB Titan Z.
• With CUDA 11.6 all sm_4x cards (Maxwell: GTX 980, etc)
• With CUDA 12 all sm_5x cards and above.
• ROCm would support all Polaris GFX804+ GCN4 cards such as the Sapphire 16GB RX 570 or 2x 16GB Radeon Pro Duo, Vega, and above.

Old GPUs may only work with their corresponding old CUDA/ROCM libraries. But even if these different CUDA/ROCM libraries do co-exist in the system, I don't think the compiler can use all of them. For example, in the code we have #include <cuda.h>, the compiler will only use the first found cuda.h. Besides, old GPUs may not be officially supported by the latest pytorch neither.

And I don't think different libraries co-existing in the system fits the philosophy of arch linux. It sounds more like gentoo.

Back to your situations, I think they are the ideal for virtual enviroments, anaconda or even docker. Just create an enviroment for each kind of your computing resources on your machine and install the right version of pytorch then somehow provide a unified interface to access the pytorch of them. For example, the users can run their code like:

enviroment=cuda # Or rocm, cuda8, cpu_only or any enviroment
source /path/to/$enviroment/bin/activate
python train.py

from arch4edu.

Thank you for the advice regarding how to proceed!

I’m also thinking of using SHARKSHARK instead to use the iGPU, dGPU, CPU together instead of containerized apart.

The SHARK approach may be a little more difficult given lack of documentation (but should be a cleaner solution! — will have to request a package for that if it works!! ;-)

from arch4edu.

This being said - is it possible to have a package for PyTorch and TensorFlow that is “opt-ROCM” or “opt-CUDA” but the “opt” part includes both Intel CPUs & GPUs & Myriad VPUs?

Aka the behavior OPENVINO has today (yet another one without a package!)

from arch4edu.

Summary so far: these are two HSA machines which need local virtual environments and a new backend to schedule/coordinate/orchestrate/facilitate the distribution of neural workloads. (Or another clever solution)

I have tried to use various existing techniques to expose virtualized environments to each other to no avail and am again limited by this lack of the "neural fabric" interface for multiple virtual environments to distribute and coordinate the processing of jobs simultaneously. I also do not yet have the Python skills to create this.

I have opened an issue on the Easy Diffusion project Github.

I have enrolled in an MIT 12-Week crash course in AI but I believe all that will do is give me a better overview before diving deeper. It will be awhile before I am able to develop the strategy, though the dev environment is lying in wait!

from arch4edu.