Comments (5)
Ah yes, I think this is what I was missing, it is now running and it looks busy as expected.
Thank you !
from ck.
Follow-up, I deleted the cache folders related to that test in CM/repos/local/cache/
and re-ran the command and it is now running as expected.
Is there a clean way to do this ?
* Tags: bert,bert-large,bert-squad,get,language,language-processing,ml-model,raw,script-artifact-5e865dbdc65949d2,_amazon-s3,_fp32,_onnx,_unpacked
Path: /root/CM/repos/local/cache/966c040426324c19
* Tags: bert,bert-large,bert-squad,get,language,language-processing,ml-model,raw,script-artifact-5e865dbdc65949d2,_amazon-s3,_int8,_onnx,_unpacked
Path: /root/CM/repos/local/cache/bebee43d0df64737
* Tags: harness,inference,mlcommons,mlperf,nvidia,nvidia-harness,reproduce,script-artifact-bc3b17fb430f4732,_batch_size.1,_bert-99,_bert_,_build_engine,_cuda,_singlestream,_tensorrt
Path: /root/CM/repos/local/cache/e0df579f9a7341e8
* Tags: harness,inference,mlcommons,mlperf,nvidia,nvidia-harness,reproduce,script-artifact-bc3b17fb430f4732,_batch_size.1024,_bert-99,_bert_,_build_engine,_cuda,_offline,_tensorrt
Path: /root/CM/repos/local/cache/7f85574449214974
from ck.
Oh, actually there is a --rerun
flag to force rerun even when results exist. Can you please try it?
from ck.
from ck.
cmr "generate-run-cmds inference _submission _all-scenarios" --model=bert-99 \
--device=cuda --implementation=nvidia-original --backend=tensorrt \
--execution-mode=valid --results_dir=$HOME/results_dir \
--category=edge --division=open --quiet --rerun
This should be the command to get a valid result. If the estimated target_qps
is incorrect you can also provide --offline_target_qps=<ACTUAL_VALUE>
to get a valid run. A run is valid only if it runs for 10 minutes and how long the benchmark run (in offline scenario) is determined by the given/estimated target_qps value.
from ck.
Related Issues (20)
- Print migration warning when using mlcommons@ck HOT 7
- Add universal check of env vars in cmind.utils HOT 1
- Add "prototype" flag to CM script meta
- `convert_path` is not part of setuptools API and will be removed HOT 7
- Refactor CM, CM for MLOps and CM for MLPerf docs and tutorials
- Improving CM core
- When dumping version info from dependencies, variations do not have _ HOT 1
- Warning Encountered During pip install cmind on Ubuntu via WSL HOT 3
- KeyError in MLPerf Inference with ResNet-50 HOT 5
- Could not identify license file for opentelemetry-cpp HOT 1
- CUDA version 12.4 not supported for this cm command HOT 1
- Support branch for cm pull repo
- How do you specify which GPU to run an Mlperf benchmark on with CM? HOT 6
- cm add script is failing on new CM repository HOT 1
- Support ssh URLs in cm pull repo HOT 3
- Improve the accessibility of the documentations HOT 1
- How to prevent caching? HOT 2
- Requiring a user access token or an SSH key instead for huggingface.co HOT 2
- Portable CM script failed (name = build-docker-image, return code = 256) HOT 1
- prototype "cm init" to check some system deps (git, wget) and pull cm4mlops repo
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ck.