Comments (5)
ieggel
commented on July 19, 2024
1
Hi @PatWie
Updated the repo in order to support the provision of the YAML config file.
Have a look here: https://github.com/ieggel/cluster-smi-docker
from cluster-smi.
PatWie
commented on July 19, 2024
I also thought about using docker. And I like docker-compose setups. But this is the beautify of Go. It is a simple portable binary. The only missing part is static linkage to libzmq.
Maybe
pebbe/zmq4#121
is worth a try.
from cluster-smi.
ieggel
commented on July 19, 2024
Having it on Docker brings several advantages:
- no installation of libraries or other tools
- no compilation
- no OS requirements
- no version hell
- reproducability
- adding a new node to a gpu cluster takes second
- setting up a cluster of 10 nodes takes around 1 minute once the image is
built
- does not pollute system
- ...
It was just a proposition for people that prefer to run it on Docker, not
pro/cons of Docker or Go.
PS: nice tool. Works great!
Cheers,
-Ivan
…On Fri, May 4, 2018, 20:18 Patrick Wieschollek ***@***.***> wrote:
I also thought about using docker. And I like docker-compose setups. But
this is the beautify of Go. It is a simple portable binary. The only
missing part is static linkage to libzmq.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#13 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIxCmmscdqcNb5Lhm5Je7GHHkadYrn28ks5tvJuMgaJpZM4Ty7C0>
.
from cluster-smi.
PatWie
commented on July 19, 2024
I had a look at your Dockerfile. I suggest to use the CLUSTER_SMI_CONFIG_PATH env-var and add the YAML config file via a volume. So it can be re-used at different setups (different ips, ports, etc) without building the entire container from scratch.
From my understanding, https://github.com/ieggel/cluster-smi-docker/blob/master/Dockerfile#L34 this dynamic linkage does not work in general, when the path changes. Can you run the produced binary at a different machine?
I will consider this dockerfile for CI-testing though.
from cluster-smi.
ieggel
commented on July 19, 2024
Hi Pat,
Regarding the config,that's definitely better than the current solution. I
will have a look as soon as i can.
Regarding PKG_CONFIG_PATH: It will work on any machine, since everything
needed is directly included in the Docker image. You say it does not work
if the path changes. OK this is true, but in a Docker image the path will
never change, since a docker image can be seen as a self-containing
(EVERYTHING needed is included in the image) read-only layer. So no matter
where the docker build is run, it will always produce the same image. Also,
running the Docker container (instance of a Docker image) will work on any
machine where docker is installed (additionally docker-nvidia needed for
gpu related stuff).
Cheers,
-Ivan
…On Fri, May 18, 2018 at 8:01 AM, Patrick Wieschollek < ***@***.***> wrote:
I had a look at your Dockerfile. I suggest to use the
CLUSTER_SMI_CONFIG_PATH env-var and add the YAML config file
<https://github.com/PatWie/cluster-smi/blob/master/cluster-smi.example.yml>
via a volume. So it can be re-used at different setups (different ips,
ports, etc) without building the entire container from scratch.
From my understanding, https://github.com/ieggel/cluster-smi-docker/blob/
master/Dockerfile#L34 this dynamic linkage does not work in general, when
the path changes. Can you run the produced binary at a different machine?
I will consider this dockerfile for CI-testing though.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#13 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIxCmncHDMrYvchAhtguCBphqWAhjJLvks5tzmOygaJpZM4Ty7C0>
.
from cluster-smi.
Related Issues (11)
- Use syncMap
- Possible deadlock? HOT 6
- Cannot compile HOT 21
- Multiple GPU cluster within same network HOT 4
- [error] make all HOT 1
- [error] build cluster-smi-router.go config.go HOT 2
- Static Linkage to libzmq to allow pre-built binaries HOT 1
- Show processes running on the GPUs HOT 8
- Add Timeout message HOT 1
- Add yaml package to godep HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cluster-smi.