When I try to use the dcgmi to monitor under A100 MIG config, I got this problem about dcgm HOT 5 CLOSED

@xichie,

dcgmi diag does not support all MIG configurations at this moment. The only supported MIG configuration is when GPU instance occupies the whole GPU and has just one Compute instance. In general - that is configuration similar to disabled MIG.

thank you！And I want to use dcgm with python to get some metrics under MIG configuration，such as sm utilization，gpu instance memory utilization etc. I am not ensure dcgm can do this. Could you give me a example？

from dcgm.

Here is an example of how to get DCP metrics for the whole GPU: test_prof.py

And here is an example of how to work with GPU/Compute instances: test_instances.py

from dcgm.

cmd: dcgmi diag -r 1
print: GPU 0's MIG configuration is incompatible with the diagnostic because it prevents access to the entire GPU.

from dcgm.

@xichie,

dcgmi diag does not support all MIG configurations at this moment. The only supported MIG configuration is when GPU instance occupies the whole GPU and has just one Compute instance. In general - that is configuration similar to disabled MIG.

from dcgm.

cmd: dcgmi diag -r 1 print: GPU 0's MIG configuration is incompatible with the diagnostic because it prevents access to the entire GPU.

Thank you so much for your help! 🙏🙏🙏

from dcgm.