Comments (3)
One new reason to split system dependencies and pypi deps is to install them separately and then merge them in one layer to accelerate the process.
func base() llb.State {
python := llb.Image("docker.io/library/python:3.8")
run1 := python.Run(llb.Shlex("pip install -i https://mirror.sjtu.edu.cn/pypi/web/simple jupyter"))
run1.AddMount("/root/.cache/pip", llb.Scratch())
run1.AddMount("/root/.cache/pip", llb.Scratch(), llb.AsPersistentCacheDir("//root/.cache/pip", llb.CacheMountShared))
run := python.
Run(llb.Shlex("pip install -i https://mirror.sjtu.edu.cn/pypi/web/simple ormb"))
run.AddMount("/root/.cache/pip", llb.Scratch())
run.AddMount("/root/.cache/pip", llb.Scratch(), llb.AsPersistentCacheDir("//root/.cache/pip", llb.CacheMountShared))
new := llb.Merge([]llb.State{run.State, run1.State})
return new
}
from envd.
One way to accelerate the process: Make a tiny image which contains pip/apt, then use the image to pip install deps. And finally merge the base image with them.
func base() llb.State {
// Maybe large!
python := llb.Image("docker.io/library/python:3.8")
run1 := some-tiny-image.Run(llb.Shlex("pip install -i https://mirror.sjtu.edu.cn/pypi/web/simple jupyter"))
run1.AddMount("/root/.cache/pip", llb.Scratch())
run1.AddMount("/root/.cache/pip", llb.Scratch(), llb.AsPersistentCacheDir("//root/.cache/pip", llb.CacheMountShared))
run := some-tiny-image.
Run(llb.Shlex("pip install -i https://mirror.sjtu.edu.cn/pypi/web/simple ormb"))
run.AddMount("/root/.cache/pip", llb.Scratch())
run.AddMount("/root/.cache/pip", llb.Scratch(), llb.AsPersistentCacheDir("//root/.cache/pip", llb.CacheMountShared))
new := llb.Merge([]llb.State{python, run.State, run1.State})
return new
}
from envd.
Let's keep the rules to:
install_package(
name = ['vscode', 'gdb>9', 'ssh', 'g++'],
optional = True
)
pip_package(
name = ['numpy', 'pandas']
)
from envd.
Related Issues (20)
- feat: Improve error message
- bug: recent changelog loses some commits HOT 3
- refactor: bootstrap buildkit config file
- feat: create an pre-defined dev env in one command
- bug: wrong binary name in CICD HOT 1
- feat: disable buildkit merge op when using moby builder HOT 2
- bug: ERROR exporting to oci image format when the first character of directory name is underline HOT 1
- bug: bootstrap failed with timeout 5s: cannot connect to buildkitd in version 0.3.36 HOT 8
- feat: support Nydus type image builder HOT 8
- bug: No linux/arm64 wheel in 0.3.37 release HOT 3
- feat: support `io.copy` from another image
- bug: envd up failed in debug mode HOT 1
- bug: bootstrap does not work in docker-from-docker setting HOT 6
- feat: support `pyproject.toml` HOT 3
- bug: use-proxy not working HOT 5
- feat: the "evnd up" command adds a gpu-set parameter that looks like "docker run --gpus" HOT 2
- feat: add `envd shutdown` command to deinitialize envd HOT 2
- bug: ERROR importing cache manifest from docker.io/tensorchord/python-cache:envd-v0.3.43 1.8s HOT 2
- BUG: Cannot build image based on envd build image HOT 1
- feat: Publish ports HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from envd.