cloud66-oss / habitus Goto Github PK
View Code? Open in Web Editor NEWA build flow tool for Docker.
Home Page: https://help.cloud66.com/habitus/
License: Apache License 2.0
A build flow tool for Docker.
Home Page: https://help.cloud66.com/habitus/
License: Apache License 2.0
When I try and run habitus (Linux build, ubuntu machine) I get the following error:
> habitus -f habitus.yaml --host tcp://docker:2345
2016/05/07 14:46:20 ▶ Using 'habitus.yaml' as build file
2016/05/07 14:46:20 ▶ open cert.pem: no such file or directory
My Docker doesn't require TLS to connect to. It looks like builder.go (https://github.com/cloud66/habitus/blob/f133be298d74c444bb260684e7b4f6989f4fdb18/build/builder.go#L59-L63) assumes it does.
The log when running multiple images in parallel seems confusing and difficult to follow.
When it gets to the actual docker steps, it becomes impossible to understand which command relates to which step.
2016/10/19 12:58:53 Parallel build for go_base
2016/10/19 12:58:53 Parallel build for frontend_api
2016/10/19 12:58:53 Building go_base
2016/10/19 12:58:53 Building frontend_api
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.go_base'
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.frontend_api'
2016/10/19 12:58:53 Writing the new Dockerfile into Dockerfile.go_base.generated
2016/10/19 12:58:53 Parallel build for hdl7 �[0m
2016/10/19 12:58:53 Building hdl7 �[0m
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.hdl7
would it be possible to tag these lines with something like the step name or number so it is easies to understand which step they refer to?
regards
Alessandro
qiuyuzhou ⋯ git.shundaojia.com servers carpoolsch habitus -version
Habitus - v0.4.6 (c) 2016 Cloud 66 Inc.
qiuyuzhou ⋯ git.shundaojia.com servers carpoolsch habitus -version
Habitus - v0.4.6 (c) 2016 Cloud 66 Inc.
qiuyuzhou ⋯ git.shundaojia.com servers carpoolsch habitus -version
Habitus - v0.4.6 (c) 2016 Cloud 66 Inc.
qiuyuzhou ⋯ git.shundaojia.com servers carpoolsch habitus -version
failed MSpanList_Insert 0x92d000 0xb0892f31fb6 0x0 0x0
fatal error: MSpanList_Insert
runtime stack:
runtime.throw(0x5e1d10, 0x10)
/Users/khash/go/src/runtime/panic.go:530 +0x90 fp=0x7fff5fbff0c0 sp=0x7fff5fbff0a8
runtime.(*mSpanList).insert(0x82d288, 0x92d000)
/Users/khash/go/src/runtime/mheap.go:933 +0x293 fp=0x7fff5fbff0f0 sp=0x7fff5fbff0c0
runtime.(*mheap).freeSpanLocked(0x82ca80, 0x92d000, 0x100, 0x0)
/Users/khash/go/src/runtime/mheap.go:809 +0x4be fp=0x7fff5fbff158 sp=0x7fff5fbff0f0
runtime.(*mheap).grow(0x82ca80, 0x8, 0x0)
/Users/khash/go/src/runtime/mheap.go:675 +0x2a0 fp=0x7fff5fbff1b0 sp=0x7fff5fbff158
runtime.(*mheap).allocSpanLocked(0x82ca80, 0x1, 0x0)
/Users/khash/go/src/runtime/mheap.go:553 +0x4e3 fp=0x7fff5fbff208 sp=0x7fff5fbff1b0
runtime.(*mheap).alloc_m(0x82ca80, 0x1, 0x15, 0x0)
/Users/khash/go/src/runtime/mheap.go:437 +0x119 fp=0x7fff5fbff238 sp=0x7fff5fbff208
runtime.(*mheap).alloc.func1()
/Users/khash/go/src/runtime/mheap.go:502 +0x41 fp=0x7fff5fbff268 sp=0x7fff5fbff238
runtime.systemstack(0x7fff5fbff288)
/Users/khash/go/src/runtime/asm_amd64.s:307 +0xab fp=0x7fff5fbff270 sp=0x7fff5fbff268
runtime.(*mheap).alloc(0x82ca80, 0x1, 0x10000000015, 0x12a6f)
/Users/khash/go/src/runtime/mheap.go:503 +0x63 fp=0x7fff5fbff2b8 sp=0x7fff5fbff270
runtime.(*mcentral).grow(0x82e680, 0x0)
/Users/khash/go/src/runtime/mcentral.go:209 +0x93 fp=0x7fff5fbff320 sp=0x7fff5fbff2b8
runtime.(*mcentral).cacheSpan(0x82e680, 0x826fa8)
/Users/khash/go/src/runtime/mcentral.go:89 +0x47d fp=0x7fff5fbff360 sp=0x7fff5fbff320
runtime.(*mcache).refill(0x929000, 0x15, 0x7fff5fbff3c8)
/Users/khash/go/src/runtime/mcache.go:119 +0xcc fp=0x7fff5fbff398 sp=0x7fff5fbff360
runtime.mallocgc.func2()
/Users/khash/go/src/runtime/malloc.go:642 +0x2b fp=0x7fff5fbff3b8 sp=0x7fff5fbff398
runtime.systemstack(0x7fff5fbff458)
/Users/khash/go/src/runtime/asm_amd64.s:307 +0xab fp=0x7fff5fbff3c0 sp=0x7fff5fbff3b8
runtime.mallocgc(0x180, 0x57bba0, 0x0, 0x800000000)
/Users/khash/go/src/runtime/malloc.go:643 +0x869 fp=0x7fff5fbff498 sp=0x7fff5fbff3c0
runtime.newobject(0x57bba0, 0x827790)
/Users/khash/go/src/runtime/malloc.go:781 +0x42 fp=0x7fff5fbff4c0 sp=0x7fff5fbff498
runtime.malg(0x8000, 0x827b40)
/Users/khash/go/src/runtime/proc.go:2634 +0x27 fp=0x7fff5fbff4f8 sp=0x7fff5fbff4c0
runtime.mpreinit(0x828300)
/Users/khash/go/src/runtime/os1_darwin.go:140 +0x1f fp=0x7fff5fbff510 sp=0x7fff5fbff4f8
runtime.mcommoninit(0x828300)
/Users/khash/go/src/runtime/proc.go:494 +0x105 fp=0x7fff5fbff558 sp=0x7fff5fbff510
runtime.schedinit()
/Users/khash/go/src/runtime/proc.go:434 +0x79 fp=0x7fff5fbff5a0 sp=0x7fff5fbff558
runtime.rt0_go(0x7fff5fbff5d8, 0x2, 0x7fff5fbff5d8, 0x0, 0x0, 0x2, 0x7fff5fbff7a8, 0x7fff5fbff7b0, 0x0, 0x7fff5fbff7b9, ...)
/Users/khash/go/src/runtime/asm_amd64.s:138 +0x132 fp=0x7fff5fbff5a8 sp=0x7fff5fbff5a0
So I wanted to test habitus and made this project. When travisci builds it, this is the output:
$ sudo wget https://github.com/cloud66/habitus/releases/download/0.4.9/habitus_linux_amd64 -o habitus
$ sudo chmod +x habitus
$ sudo ./habitus -version
./habitus: 1: ./habitus: --2017-02-19: not found
./habitus: 2: ./habitus: Syntax error: "(" unexpected
The command "sudo ./habitus -version" failed and exited with 2 during .
Your build has been stopped.
When I run habitus, I get the following error
$ sudo habitus_linux_amd64
2016/10/24 12:16:43 ▶ Using '/home/wael/myProject/build.yml' as build file
2016/10/24 12:16:43 ▶ open cert.pem: no such file or directory
Environment:
Docker is installed locally and does not require TLS for connection
When I try building an image on an OS X El Capitan host, the following error is thrown once the squash
step is reached:
2016/09/01 23:52:01 ▶ Loading export from /tmp/habitus-export-527359243 using /tmp/docker-squash612669813 for tempdir
2016/09/01 23:52:02 ▶ Loaded image w/ 18 layers
2016/09/01 23:52:02 ▶ Extracting layers...
2016/09/01 23:52:02 ▶ - /tmp/docker-squash612669813/036fc864e6c7d0b6f75fe523e4a1a852f72115062e76f0a9d14877989948eb4a/layer.tar
tar: Option --xattrs is not supported
Usage:
List: tar -tf <archive-filename>
Extract: tar -xf <archive-filename>
Create: tar -cf <archive-filename> [filenames...]
Help: tar --help
2016/09/01 23:52:02 ▶ Build for step carrier failed due to exit status 1
I am fairly sure this line is causing the problem however I am not knowledgeable enough in this area to actually resolve the problem.
Related issue on docker-squash
Dirty work-around, I have tested and this does work however it feels like there should be a neater solution to this problem
Is there a way to do 'docker push' for example after building the image.
I have a custom script that does 'docker build' and would like to push the image to a private repository for example. I handle this via a shell script. Habitus being a build tool, would be nice to have a section that lists commands that executes outside of the container after each build step so I can get rid of the shell script.
This would be hugely helpful in the case of java maven builds.
I'm familiar with the trick of copying over just the pom.xml
(not forgetting your submodules) and running mvn dependency:resolve
(demonstrated nicely here). Changes to pom.xml
files are common enough occurrences, however, and in my case a change to a pom incurs a ~200mb download. Since my CI is hosted in the cloud that's a lot of network usage, which can get $$$.
If the dependencies were instead exposed in a volume, they would be downloaded only once for the lifetime of the project.
Am I correct to assume that this project is currently not under development?
The description for the --keep-all
option seems to suggest the existence of a keep
flag for individual steps that would prevent image removal for intermediate steps:
flag.BoolVar(&config.KeepSteps, "keep-all", false, "Overrides the keep flag for all steps. Used for debugging")
Does such a flag actually exist?
(using version 0.4.10)
Is this possible? eg using the build.yml
file from the docs as an example:
build:
version: 2016-03-14 # version of the build schema.
steps:
builder:
name: builder
dockerfile: Dockerfile.builder
# snip
deployment:
name: ironmountain
dockerfile: Dockerfile.deployment
depends_on:
- builder
uploader:
name: uploader
dockerfile: Dockerfile.uploader
depends_on:
- ironmountain
# snip
I don't necessarily want to run the uploader
every time I want to run habitus; most of the time I'm interested in only the builder
step. Is there a way to specify that step only?
Currently just has a title. Needs basic description, instructions on running, caveats (if any), etc.
I'm trying to use the secrets feature however my build is hanging when connecting to port 8080
Command ./habitus_darwin_amd64 -f habitus.yaml --build host=192.168.99.1
Dockerfile:
FROM alpine:3.2
ARG host
RUN TOKEN=$(wget http://$host:8080/api/v1/secrets/id_rsa) && echo 'ENV:' && env && echo 'TOKEN ENV' && echo $TOKEN && unset $TOKEN
RUN wget -O ~/.ssh/id_rsa http://$host:8080/api/v1/secrets/id_rsa && ssh -T [email protected] && rm ~/.ssh/id_rsa
Build YAML
build:
version: 2016-03-14
steps:
builder:
name: builder
dockerfile: Dockerfile-habitus
secrets:
id_rsa:
type: file
value: _env(HOME)/.ssh/id_rsa
Output:
tep 1 : FROM alpine:3.2
---> 558af09712a4
Step 2 : ARG host
---> Using cache
---> a20b789279a3
Step 3 : RUN TOKEN=$(wget http://$host:8080/api/v1/secrets/id_rsa) && echo 'ENV:' && env && echo 'TOKEN ENV' && echo $TOKEN && unset $TOKEN
---> Running in 43d34c9d7d1c
Connecting to 192.168.99.1:8080 (192.168.99.1:8080)
In a separate terminal window, if I try and access the resource then I get a 404.
curl 192.168.99.1:8080/api/v1/secrets/id_rsa
{"Error":"Resource not found"}%
Any ideas?
Is there a way to collect a whole directory?
Example, with Elixir releases we get a directory that represents the artifact /rel
, passing this to artifacts in the build.yml
results in an 'Invalid Header' error.
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["ls"]
gets modified to
entrypoint /docker-entrypoint.sh
CMD ["ls"]
Is that an expected behaviour?
Hello,
I'm using Habitus with this yml file
build:
version: 2016-03-14
steps:
builder:
name: builder
dockerfile: Dockerfile.builder
- /home/MainUser/test.tar.gz
deploy:
name: deploying
dockerfile: Dockerfile.deploy
When I run it, I get following error:
2017/04/20 15:08:05 notifying bugsnag: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U`
2017/04/20 15:08:05 ? Collecting artifact information
panic: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U` [recovered]
panic: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U`
goroutine 1 [running]:
panic(0x7dcae0, 0xc0420f7720)
/usr/local/go/src/runtime/panic.go:500 +0x1af
github.com/cloud66/habitus/vendor/github.com/bugsnag/bugsnag-go.AutoNotify(0xc0420f7740, 0x1, 0x1)
/usr/local/go/src/github.com/cloud66/habitus/vendor/github.com/bugsnag/bugsnag-go/bugsnag.go:52 +0x13a
panic(0x7dcae0, 0xc0420f7720)
/usr/local/go/src/runtime/panic.go:458 +0x251
regexp.MustCompile(0xc0420f3440, 0x30, 0xc042093030)
/usr/local/go/src/regexp/regexp.go:237 +0x181
github.com/cloud66/habitus/build.(*Builder).collectHostArtifactRoots(0xc042072780, 0xc0420f76b0, 0x1, 0x1)
/usr/local/go/src/github.com/cloud66/habitus/build/builder.go:184 +0x30d
github.com/cloud66/habitus/build.(*Builder).StartBuild(0xc042072780, 0xc042086d80, 0xc042072780)
/usr/local/go/src/github.com/cloud66/habitus/build/builder.go:109 +0xd49
main.main()
/usr/local/go/src/github.com/cloud66/habitus/main.go:149 +0xe66
2017/04/20 15:08:05 notifying bugsnag: regexp: Compile(`^C:\Users\Pawel\Documents\Docker\dockerKinetis/+`): error parsing regexp: invalid escape sequence: `\U`
[recovered]
I don't know what I am doing wrong. Without artifacts everything is OK.
Having the following (simplified) build.yml:
build:
version: 2016-03-14
steps:
builder:
name: builder
dockerfile: builder/Dockerfile
And when using the following command:
sudo habitus --host=unix:///var/run/docker.sock --binding=127.0.0.1 --keep-all --no-cache -f builder/build.yml
(and probably a simpler version of it as well, didn't try out)
the following occurs:
2017/01/06 17:00:49 ▶ Using 'builder/build.yml' as build file
2017/01/06 17:00:49 ▶ Collecting artifact information
2017/01/06 17:00:49 ▶ Building 2 steps
2017/01/06 17:00:49 ▶ Starting API on 8080
2017/01/06 17:00:49 ▶ Step 0 - builder: builder
2017/01/06 17:00:49 ▶ Step 1 - app-base: app-base
2017/01/06 17:00:49 ▶ Parallel build for builder
2017/01/06 17:00:49 ▶ Building builder
2017/01/06 17:00:49 ▶ Parsing and converting 'builder/Dockerfile'
2017/01/06 17:00:49 ▶ Writing the new Dockerfile into builder/Dockerfile.generated
2017/01/06 17:00:49 ▶ Building the builder image from Dockerfile.generated
2017/01/06 17:00:49 ▶ Build for step builder failed due to API error (500): {"message":"Cannot locate specified Dockerfile: Dockerfile.generated"}
We can clearly see in the above log that Habitus is writing to builder/Dockerfile.generated
but then tries to read from Dockerfile.generated
instead.
Being able to run Habitus from a parent directory using the -f
argument would allow to share resources (like configuration files) in a common top-level folder.
In the case below: local/central-b3589d5d7afb365fe9978c9f0f6741d8a1baaaa4:latest
was incorrectly identified to be removed (even though it was the last step)
---> e02789b39a5d
Removing intermediate container d0075de08c28
Successfully built e02789b39a5d
2017/03/22 12:43:24 ▶ Removing artifact path: /var/lib/buildkite-agent/builds/buildkite-1/cloud-66-1/continuous-build/app-artifact.tar.gz
2017/03/22 12:43:24 ▶ Removing unwanted image local/central-b3589d5d7afb365fe9978c9f0f6741d8a1baaaa4:latest
2017/03/22 12:43:24 ▶ Removing unwanted image base-b3589d5d7afb365fe9978c9f0f6741d8a1baaaa4
./main.go:72: cannot use *log (type "github.com/op/go-logging".Logger) as type "github.com/cloud66/habitus/vendor/github.com/op/go-logging".Logger in assignment
From:
go get
then
go build
hello, I have a simple setup like the following
build:
version: 2016-03-14 # version of the build schema.
steps:
builder:
name: b.gcr.io/test/go_base:_env(GIT_LABEL)
dockerfile: Dockerfile.go_base
artifacts:
- /comp3
- /comp2
- /comp1
comp3:
name: b.gcr.io/test/comp3:_env(GIT_LABEL)
dockerfile: Dockerfile.comp3
depends_on:
- builder
comp2:
name: b.gcr.io/test/comp2:_env(GIT_LABEL)
dockerfile: Dockerfile.comp2
depends_on:
- builder
comp1:
name: b.gcr.io/test/comp1:_env(GIT_LABEL)
dockerfile: Dockerfile.comp1
depends_on:
- builder
with a sample Dockerfile.comp1 as simple as:
FROM ubuntu
ADD /artifacts/comp1 /comp1
ENTRYPOINT ["/comp1"]
CMD ["--config", "/configs.yml"]
When I run habitus
, I get the following error
2016/10/11 16:36:41 ▶ Build for step b.gcr.io/test/comp1:fix_test-ademaria failed due to Unknown flag: config
hello, I have a simple setup like the following
build:
version: 2016-03-14 # version of the build schema.
steps:
builder:
name: b.gcr.io/test/go_base:_env(GIT_LABEL)
dockerfile: Dockerfile.go_base
artifacts:
- /comp3
- /comp2
- /comp1
comp3:
name: b.gcr.io/test/comp3:_env(GIT_LABEL)
dockerfile: Dockerfile.comp3
depends_on:
- builder
comp2:
name: b.gcr.io/test/comp2:_env(GIT_LABEL)
dockerfile: Dockerfile.comp2
depends_on:
- builder
comp1:
name: b.gcr.io/test/comp1:_env(GIT_LABEL)
dockerfile: Dockerfile.comp1
depends_on:
- builder
with a sample Dockerfile.comp1 as simple as:
FROM ubuntu
ADD ./comp1 /comp1
CMD ["/comp1", "--config", "/configs.yml"]
When I run habitus, the images are always removed:
2016/10/11 16:33:20 ▶ Removing unwanted image b.gcr.io/test/comp1:fix_test-ademaria
2016/10/11 16:33:20 ▶ Removing unwanted image b.gcr.io/test/comp2:fix_test-ademaria
2016/10/11 16:33:20 ▶ Removing unwanted image b.gcr.io/test/comp3:fix_test-ademaria
How can I tell habitus that I want to keep an image?
How do I push it to a repository once built?
Currently, the stat command is used before extracting files from the container (see: https://github.com/cloud66/habitus/blob/master/build/builder.go#L340). Certain containers (such as
scratch`) will not contain this method and will fail to extract the file. In those cases it may make sense to fallback to not copying over the file permissions.
ENV MAVEN_HOME=/opt/maven PATH=$MAVEN_HOME/bin/:$PATH
gets converted into -
env MAVEN_HOME /opt/maven PATH $MAVEN_HOME/bin/:$PATH
should have been -
env MAVEN_HOME=/opt/maven PATH=$MAVEN_HOME/bin/:$PATH
Reference: https://docs.docker.com/engine/reference/builder/#/env
There are some command line flags (such as no-cache
) which may make more sense to be on each job in the yml
, allowing for greater customization
I get random errors about images missing when builds run in parallel (with the uid flag set)
Hello
It seems that if I use CMD ["/my_binary"] in my Docker.deployment it is considered as if i typed: CMD /my_binary
This is what i expect when running docker inspect:
"Cmd": [
"/main"
],
and this is what i get using habitus:
"Cmd": [
"/bin/sh",
"-c",
"/main"
],
If i use docker build with the exact same dockerfile it works as expected.
I did an example to reproduce the issue:
https://github.com/sebest-blog/create-a-small-docker-image-for-a-golang-binary-with-habitus
Hello,
I have a set of images that depend on an image "go_base".
I am affected by something similar to #34 reported by @kamerblauwlicht
I am running 0.4.7 and I still see this happening.
2016/10/19 12:58:53 Parallel build for go_base
2016/10/19 12:58:53 Building go_base
2016/10/19 12:58:53 Parsing and converting 'Dockerfile.go_base'
2016/10/19 12:58:53 Building the go_base-66b52139 image from Dockerfile.go_base.generated
[cut]
2016/10/19 12:59:07 Writing the new Dockerfile into Dockerfile.message_store.generated
2016/10/19 12:59:07 Building the go_message_store-66b52139 image from Dockerfile.message_store.generated
Step 1 : FROM go_base
Step 1 : FROM go_base
Pulling repository docker.io/library/go_base
Pulling repository docker.io/library/go_base
2016/10/19 12:59:12 Build for step go_message_store failed due to Error: image library/go_base:latest not found
Build step 'Execute shell' marked build as failure
this is intermittent and re-running the job usually fixes it.
The documentation for the no-cache
flag (https://github.com/cloud66/habitus/blob/master/main.go#L43) describes it as enabled caching, not disabling it
hello, I have a simple setup like the following
build:
version: 2016-03-14 # version of the build schema.
steps:
builder:
name: _env(IMAGE_PREFIX)/go_base:_env(GIT_LABEL)
dockerfile: Dockerfile.go_base
artifacts:
- /comp3
- /comp2
- /comp1
When I run it with
habitus -f build2.yml -env GIT_LABEL=label -env IMAGE_PREFIX=prefix
I get the following error:
2016/10/11 16:41:58 ▶ Using 'build2.yml' as build file
2016/10/11 16:41:58 ▶ Building 1 steps
2016/10/11 16:41:58 ▶ Step 0 - builder:
2016/10/11 16:41:58 ▶ Parallel build for
2016/10/11 16:41:58 ▶ Building
2016/10/11 16:41:58 ▶ Parsing and converting 'Dockerfile.go_base'
2016/10/11 16:41:58 ▶ Starting API on 8080
2016/10/11 16:41:58 ▶ Writing the new Dockerfile into Dockerfile.go_base.generated
2016/10/11 16:41:58 ▶ Building the image from Dockerfile.go_base.generated
sha256:ec15bedfbd0cb2c189099c16a8d92c0aeac1caedd18724567451df61a17a1413
2016/10/11 16:42:00 ▶ Building container based on the image
2016/10/11 16:42:00 ▶ Build for step failed due to API error (500): {"message":"Invalid container name (.8tf0SKYpExrgdLvd), only [a-zA-Z0-9][a-zA-Z0-9_.-] are allowed"}
so it looks as if the _env(VAR)
directive in this case is not evaluated
I spent some time figuring this out so I hope it can help others or be added to examples.
Dockerfile
FROM ubuntu
RUN apt-get update && apt-get install -y wget openssh-client
# add the authorized host key for github (avoids "Host key verification failed")
RUN mkdir ~/.ssh && ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
ARG host
ENV PRIVATE_KEY /root/.ssh/id_rsa
RUN wget -O $PRIVATE_KEY http://$host:8080/v1/secrets/file/id_rsa \
&& chmod 0600 $PRIVATE_KEY \
&& ssh -T [email protected] \
&& rm $PRIVATE_KEY
build.yml
build:
version: 2016-03-14
steps:
builder:
name: builder
dockerfile: Dockerfile
secrets:
id_rsa:
type: file
value: _env(HOME)/.ssh/id_rsa
Run it (OSX command line)
habitus --build host=$(ipconfig getifaddr en0) --host unix:///var/run/docker.sock --binding=0.0.0.0
artefacts
is the British spelling and appears quite confusing to Americans; while both are correct, it might be simpler to use the accepted American style simply due to the other projects doing so.
I just have switched to Docker for Mac and I try to figure out how to use Habitus. So far without any luck.
➜ habitus
2016/05/22 20:34:43 ▶ Using '/path/to/build.yml' as build file
2016/05/22 20:34:43 ▶ open cert.pem: no such file or directory
➜ habitus --use-tls=false
2016/05/22 20:34:50 ▶ Using '/path/to/build.yml' as build file
2016/05/22 20:34:50 ▶ invalid endpoint
I've noticed that I'm running habitus pretty regularly like so:
sudo habitus --host=unix:///var/run/docker.sock --binding=127.0.0.1
So here are my thoughts:
It would be nice if --host
defaulted to the the most common value unix:///var/run/docker.sock
if the DOCKER_HOST
env variable is not present. If there is no socket file or ENV variable, fallback to today's behavior.
Similarly, it would make sense for the binding value to default to a (safe) local value of 127.0.0.1
by default. I could see an argument to use 0.0.0.0
(which would not be preferred for security reasons) or discover the docker bridge IP dynamically.
This would at least get the most common invocations down to just sudo habitus
.
Hey guys!
Do you want us to file issues and feature requests as proposal issues or just send PRs or something else?
Thanks!
Joey
sometimes the build is done but the image is not proper tagged.
Attempting to run without sudo yields:
Some of the build steps require admin privileges (sudo). Please run with sudo
The reason I'm trying to use habitus is for a continuous integration process that needs a private key (for github access) within the docker build. The automated CI process does not have sudo
on the host, nor do I want to grant such privileges.
I am new to habitus so I could be missing something. Can I accomplish this without sudo
while using habitus
?
-host string
must set to unix:///var/run/docker.sock
instead unix://var/run/docker.sock
Trying to run habitus even with the provided example, I get:
security git:(master) ✗ habitus_linux_amd64 --host=unix:///var/run/docker.sock --binding=127.0.0.1 --build host=127.0.0.1
2016/10/24 16:14:55 ▶ Using '/home/wael/workspace/habitus/examples/security/build.yml' as build file
2016/10/24 16:14:55 ▶ Collecting artifact information
2016/10/24 16:14:55 ▶ Building 1 steps
2016/10/24 16:14:55 ▶ Step 0 - builder: builder
2016/10/24 16:14:55 ▶ Parallel build for builder
2016/10/24 16:14:55 ▶ Building builder
2016/10/24 16:14:55 ▶ Parsing and converting 'Dockerfile'
2016/10/24 16:14:55 ▶ Starting API on 8080
2016/10/24 16:14:55 ▶ Writing the new Dockerfile into Dockerfile.generated
2016/10/24 16:14:55 ▶ Building the builder image from Dockerfile.generated
Step 1 : FROM ubuntu
---> f753707788c5
Step 2 : RUN apt-get update && apt-get install -y wget openssh-client
---> Using cache
---> f149286ec705
Step 3 : RUN mkdir ~/.ssh && ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
---> Using cache
---> d093edca0423
Step 4 : ARG host
---> Using cache
---> e744a38e08e2
Step 5 : ENV PRIVATE_KEY /root/.ssh/id_rsa
---> Using cache
---> 1a1212d088d7
Step 6 : RUN wget -O $PRIVATE_KEY http://$host:8080/v1/secrets/file/id_rsa && chmod 0600 $PRIVATE_KEY && ssh -T [email protected] && rm $PRIVATE_KEY
---> Running in 0bd430f3b4ef
--2016-10-24 14:14:55-- http://127.0.0.1:8080/v1/secrets/file/id_rsa
Connecting to 127.0.0.1:8080... failed: Connection refused.
2016/10/24 16:14:56 ▶ Build for step builder failed due to The command '/bin/sh -c wget -O $PRIVATE_KEY http://$host:8080/v1/secrets/file/id_rsa && chmod 0600 $PRIVATE_KEY && ssh -T [email protected] && rm $PRIVATE_KEY' returned a non-zero code: 4
Command
$ habitus_linux_amd64 --host=unix:///var/run/docker.sock --binding=127.0.0.1 --build host=127.0.0.1
Environment:
Our building environment is completely isolated and defining our secrets as env variables on the host is the easiest way for us to manage build time secrets. It would be great to see environment variables implemented as a secret type.
Hello folks. You list your slack team on the git readme but it doesn't let people signup unless they have been approved.
The following URL is meant to allow signup:
It would be nice to declare a directory of artifacts (such as a ruby bundle directory) instead of listing all files explicitly... for instance:
build:
version: 2016-03-14
steps:
builder:
name: myapp/builder
dockerfile: Dockerfile
artifacts:
- /bundler
....
where /bundler
contains sub directories:
bin build_info bundler cache doc extensions gems specifications
and those sub directories have files... etc.
When I visit "http://www.habitus.io"
And I click on "Instructions for OSX"
Then I receive a "HTTP/1.1 404 Not Found"
In our Dockerfile, we have some steps that we do execute using another user than root, resulting in the cleanup command rm -rf /root/.ssh/
not to execute:
rm: cannot remove '/root/.ssh/': Permission denied
Easy fix was to have command USER root
as the last entry of our Dockerfile, but that's a temporary hack 😄
As soon as I use --env variables I can no longer access secret files. When I remove the environment variables it works.
I might be missing it, but have been looking at docs for a while and still unable to find any instructions on how to install for linux. Maybe docs could take the following format:
Step 1: Install
Type this command to install blah blah blah
Step 2: Use
...
Currently hard to know how to quickly get everything configured
could you please give the steps to install and run habitus on mac os?
update go-dockerclient
fsouza/go-dockerclient#537
when you run a build, the environment variables of a parent process are not inherit.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.