option to prefetch the data about ck HOT 8 CLOSED

Yes, all CM scripts are modular and so we can do this. For example, the below command will download the full imagenet validation set and exports the downloaded paths.

cm run script - -tags=get,dataset,val,imagenet,original,_full -j

from ck.

Thank you !
I tried it but it is complaining, am I missing something ?

$ cm run script - -tags=get,dataset,val,imagenet,original, _full -j
* cm run script _full --tags="get,dataset,val,imagenet,original"

CM error: no scripts were found with above tags (when variations ignored)!

from ck.

Ok, the space before _full had to be removed, that one is working now.

But I am trying this example and nothing is happening, do you know why or how I can debug further ?
https://github.com/mlcommons/ck/tree/master/cm-mlops/script/get-ml-model-retinanet#cm-cli

$ cm run script --tags=get,ml-model,raw,resnext50,retinanet,object-detection
* cm run script "get ml-model raw resnext50 retinanet object-detection"

Thanks !

from ck.

Hi @jdesfossez,

CM scripts installs artifacts to the CM cache and make them available to other CM scripts via API and/or ENV variables.

You can see the cache with all artifacts including above model as follows:

cm show cache
cm show cache --tags=get,ml-model,resnext50

You can find your model and extra CM meta files as follows:

cm find cache --tags=get,ml-model,raw,resnext50,retinanet,object-detection

Basically CM is a database of objects connected by tags, UIDs and ENV variables ...

Please check these 2 tutorials that may give you more ideas behind CM:

• https://github.com/mlcommons/ck/blob/master/docs/tutorials/modular-image-classification.md
• https://github.com/mlcommons/ck/blob/master/docs/tutorials/sc22-scc-mlperf.md

That's how we reuse individual CM scripts (and workflows assembled from those scripts) for reproducibility initiatives at conferences and other initiatives to make it easier to run AI on different platforms ...

We are interested to know your use cases and how CM can help - please feel free to talk to us via Discord server or we can set up a conf-call ...

Thank you for your interest and feedback!

from ck.

Hi !
Thank you, that helps a lot, I missed the first tutorial, I am glad you linked it here.

My current goal is to automate performance testing of GPUs in a public cloud environment. I need to easily and quickly compare the impact of various hypervisor-level changes, so this project seems perfect for that purpose. Eventually I will use it as well to submit results.
Another quick question, is there a clean way for me to specify at run-time the location of the data ? For example if I wanted to make a local mirror and have the VM download from there instead of hitting the public servers.
Thanks again !

from ck.

Sorry @jdesfossez for the typo -- I was typing on mobile :(

"is there a clean way for me to specify at run-time the location of the data "

I believe you want to use a private URL here right? Currently we are supporting multiple downloaded sources like this but not custom URLs - we can do this by next release.

But for most of the large datasets, there is an option to provide the path to it via input/env variable and prevent a download like done here

from ck.

This solution can work for using custom URLs.

from ck.

ah perfect, thank you so much !
I will close this as it's not really an issue, but I appreciate the guidance !

from ck.