fredtingaud / bench-runner Goto Github PK

I have this error when I run the scripts.

Error or timeout

docker: invalid reference format: repository name must be lowercase.
See 'docker run --help'.

Fix this error on local but still persist on website #3

Tested on Manjaro Linux and Ubuntu 21.10.

Since quick-bench does not support including 3rd party libraries(like Godbolt does) via github links(FredTingaud/quick-bench-back-end#5), I was wondering what options existed to set this up locally?

Based on my understanding the way quick-bench works is to download the docker image from within a bench-runner image container corresponding to the compiler selected on the frontend locally by mounting docker.sock. So the only ways I can see to make a 3rd party library available is:

• ssh into a container corresponding to the compiler I want to benchmark with(eg. fredtingaud/quick-bench:clang-10.0), build my 3rd party dependency there and then run ./quick-bench.
• Similar to the above, build a new Dockerfile using fredtingaud/quick-bench:clang-10.0 as a base image and have RUN <steps to clone and build my 3rd party library> as a command. I'll also need to set the built image name and tag appropriately.

I'm curious if a simpler approach to this is possible? For example a way for the user to supply a the build step for the 3rd party library as a shell script and supported compilers to the ./quick-bench script and it set these up. Alternately implementing support for the issue mentioned above.

Windows does not came with /var/run/docker.sock how can we avoid this?

docker run -p 4000:4000 -v "$(pwd)/data:/data" --env-file local.env -e "BENCH_ROOT=$(pwd)" -it fredtingaud/bench-runner ./start-build-bench
Listening to commands on port 4000
(node:10) UnhandledPromiseRejectionWarning: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

(Use `node --trace-warnings ...` to show where the warning was created)
(node:10) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:10) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

The software starts at port 4000 but with no compiler and when I try to download one, it stays in a blank page forever...

Hi, I am running Ubuntu 21.10 and cloned the repository.

After doing

git clone https://github.com/FredTingaud/bench-runner.git
cd bench-runner
./quick-bench

I get:

docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/create": dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.

I found elsewhere (https://www.digitalocean.com/community/questions/how-to-fix-docker-got-permission-denied-while-trying-to-connect-to-the-docker-daemon-socket) that I need to do this first:

sudo chmod 666 /var/run/docker.sock

Maybe you can consider adding that to the instructions, or at least say a few keywords about the Docker socket.

After that you can also mention that you need to point the browser to localhost:4000, or whatever is in the output of the ./quickbench command.

It would be also nice to give some instructions on how to install third-party (apt) libraries although that is another question. (I followed the ssh method in #2, but QB still couldn't find the header file).

Thanks for making quick-bench/build-bench locally available via this repository.

Problem Summary

Description

The launcher script seems to be failing on timeout errors with some compilers(eg. clang10). Below are the logs when running ./quick-bench. For another compiler(gcc-10.1) I hit the same error as shown in the console log initially, but it finished downloading the base docker image because of which the 2nd time around it worked correctly.

In summary, is there a way to increase the timeout for the setup to complete?

Workaround

Manually run docker pull fredtingaud/bench-runner:<compiler-version> - can find the exact name from docker registry. Then launch ./quick-bench.

Benchmark Run

I am currently running the default provided StringCreation vs. StringCopy benchmark with no modifications.

Machine

MacOS Mojave 10.14.6

Console Log

Bench 78fe34458eb30cafde24575248f04835ad1aa533 {"host":"localhost:4000","connection":"keep-alive","content-length":"717","accept":"application/json","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36","content-type":"application/json","origin":"http://localhost:4000","sec-fetch-site":"same-origin","sec-fetch-mode":"cors","sec-fetch-dest":"empty","referer":"http://localhost:4000/q/zuXLFZS9YicLPzAMVvVRV1Z37a4","accept-encoding":"gzip, deflate, br","accept-language":"en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7"} < {"protocolVersion":4,"isAnnotated":true,"compiler":"clang-10.0","optim":"3","cppVersion":"20","lib":"gnu"}
/data/78/78fe34458eb30cafde24575248f04835ad1aa533: 60005.474ms
Bench failed /data/78/78fe34458eb30cafde24575248f04835ad1aa533

QuickBench UI Log

Error or timeout

Unable to find image 'fredtingaud/quick-bench:clang-10.0' locally
clang-10.0: Pulling from fredtingaud/quick-bench
6a5697faee43: Already exists
ba13d3bc422b: Already exists
a254829d9e55: Already exists
e741d7109288: Pulling fs layer
6993299e5f02: Pulling fs layer
5b4a5a5fa89c: Pulling fs layer
c0ce3ee15575: Pulling fs layer
a75ee2b72005: Pulling fs layer
ef22372d3668: Pulling fs layer
2ba85933b39c: Pulling fs layer
1b0b152886c1: Pulling fs layer
b6d179b485ef: Pulling fs layer
c655ed8e0d05: Pulling fs layer
c60baa820293: Pulling fs layer
2c4c3a8800d1: Pulling fs layer
60184dd3a102: Pulling fs layer
9c582296fc71: Pulling fs layer
2658b4ef7cf7: Pulling fs layer
72c07870fed4: Pulling fs layer
53957f47fdf1: Pulling fs layer
b6d179b485ef: Waiting
c655ed8e0d05: Waiting
c60baa820293: Waiting
2c4c3a8800d1: Waiting
60184dd3a102: Waiting
9c582296fc71: Waiting
c0ce3ee15575: Waiting
2658b4ef7cf7: Waiting
a75ee2b72005: Waiting
72c07870fed4: Waiting
ef22372d3668: Waiting
53957f47fdf1: Waiting
2ba85933b39c: Waiting
1b0b152886c1: Waiting
6993299e5f02: Verifying Checksum
6993299e5f02: Download complete
c0ce3ee15575: Verifying Checksum
c0ce3ee15575: Download complete
a75ee2b72005: Download complete
ef22372d3668: Verifying Checksum
ef22372d3668: Download complete
2ba85933b39c: Verifying Checksum
2ba85933b39c: Download complete
1b0b152886c1: Download complete
b6d179b485ef: Verifying Checksum
b6d179b485ef: Download complete
c655ed8e0d05: Verifying Checksum
c655ed8e0d05: Download complete
c60baa820293: Download complete
2c4c3a8800d1: Verifying Checksum
2c4c3a8800d1: Download complete
60184dd3a102: Verifying Checksum
60184dd3a102: Download complete
9c582296fc71: Verifying Checksum
9c582296fc71: Download complete
2658b4ef7cf7: Verifying Checksum
2658b4ef7cf7: Download complete
72c07870fed4: Verifying Checksum
72c07870fed4: Download complete
53957f47fdf1: Download complete
e741d7109288: Download complete
e741d7109288: Pull complete
6993299e5f02: Pull complete

I have C++ code on my server. How to use this tool to collect my code performance on my server?

I have a code which has dataset of around 200, and trying to benchmark 4 approaches to plot a comparison. This has large number of runs, so it is not giving me results in graph.

Suggest a way please for the same or consider as enhancement.