Helpful functions for tracking GPU/CPU usage in SLURM managed clusters.

Clone the repository to your home directory or otherwise.

Add the following lines to your .bashrc, change the location of the .gpus_users.bashrc file based on where you cloned the repository. The paths to each .awk file in gpus_users.bashrc will also have to be modified if a different clone directory is chosen.

if [ -f ~/slurm_usage_utils/gpus_users.bashrc ]; then
    . ~/slurm_usage_utils/gpus_users.bashrc
fi

In the following files, change the @include mycolors.awk line so that it has the absolute path of your mycolors.awk file from this repository:

lab_usage.awk
lab_usage_qos.awk
lab_usage_verbose.awk
node_usage.awk

Reload your .bashrc with source ~/.bashrc.

The following commands are currently supported

# Summary of GPU/CPU usage
gpus_users
gpus_users -q
gpus_users -v

# Summary of node usage
node_usage