transmode / gradle-docker Goto Github PK

The examples presented e.g. in the documentation don't work. Specifically, the code

task docker(type: Docker) {
  applicationName = 'nginx'
  dockerfile {
      run 'echo "deb http://archive.ubuntu.com/ubuntu precise main universe" > /etc/apt/sources.list'
      run 'apt-get update'
      run 'apt-get install -y inotify-tools nginx apache2 openssh-server'
  }
}

fails on dockerfile because the DSL was not present(?) in 1.2.

I'm using boot2docker on MacOSX and I was getting this error: Docker execution failed

Command line [docker build -t pixync/pixync-web:latest /Users/sarbogast/dev/pixync-web/build/docker] returned:
time="2015-02-25T11:04:24+01:00" level="fatal" msg="Post http:///var/run/docker.sock/v1.17/build? dockerfile=Dockerfile&rm=1&t=pixync%2Fpixync-web%3Alatest: dial unix /var/run/docker.sock: no such file or directory"

So I added useApi true, but now I'm getting another error:

> org.apache.http.conn.HttpHostConnectException: Connection to http://localhost:2375 refused

I'm a Docker newbie so I'm still lost

Just to have some context because saving an docker image is not really common, my use case was to build a docker image on a CI an deliver it as a binary instead of pushing it to a docker repository.

Since i needed it fast I made my plugin as mentioned before.
I don't know if some things can be taken from that but in case here is the link followed by some highlights:
https://github.com/baptistemesta/docker-build

The part that were important for me was a minimal configuration of the project using the plugin.
Here is a sample configuration:

apply plugin: 'distribution'
apply plugin: 'docker-build'

group = "com.myproject"
version = "1.0.0-SNAPSHOT"

docker {
    serverUrl = System.getProperty("docker.url")
    saveImageTo = new File(buildDir, "docker-image.tar")
}
distributions {
    main {
        contents {
            from { docker.saveImageTo }
        }
    }
}
tasks.distZip.dependsOn(tasks.buildDockerImage)

and to build the image you just have to run on the project

locally gradle distZip or with a remote docker gradle distZip -Ddocker.url=DOCKER_URL

I don't know if something like this can be also done in this plugin since it was more oriented on creating a docker image to run

Hi, this is just a question.

I'm trying to customize the tar that is extracted into the container. Specifically, I want to change the name of the root directory.

I'm not quite sure how to do that for only the tar that is used for the container creation.

I've tried adding the following task to my build.gradle file:

    //...
    apply plugin: 'application'
    apply plugin: 'docker'
    ///...
    task gortoolsDocker(type: Docker) {
        applicationName = "gor-tools"
        distributions {
            main {
                contents {
                    into '/gor/'
                }
            }
        }
    }

This does change the root directory in the tar file that is extracted into the container, however, it also changes the root directory in the tar file that gets created when I call ./gradlew distTar which is something that I don't want to do.

Is there any way that I can modify just the "docker tar"?

Removing intermediate images should be as easy as adding "--rm=true" to the command line of "docker build".

I'm running a private docker registry 2.0 proxied by nginx. To upload data HTTP basic authentication is required.
For the build I am using the native client. Is it possible to push data to the repository using basic authentication (docker login .... )?
As far as I can see it is not available in the sources. Is this feature planned?

Example:

task buildImage(type: Docker) {
  buildImage.dependsOn('prepBuildImage')
  dockerfile 'Dockerfile'
  tag = 'colinbjohnson/test'
  tagVersion = '1.0'
  push true
  apiUsername = 'x'
  apiPassword = 'x'
  apiEmail = 'x'
}

Returns:

Execution failed for task ':buildImage'.
> Docker API error: Failed to build Image:
  Invalid repository name (test:1.0), only [a-z0-9-_.] are allowed

I'm not sure exactly what is going on - when using curl to push (example: curl -X POST -H "X-Registry-Auth: x" http://192.168.2.2:2375/images/colinbjohnson/test/push) I'm able to push successfully, but when pushing through gradle the URL is POST http://192.168.2.2:2375/v1.11/images/colinbjohnson/test:latest/push or POST http://192.168.2.2:2375/v1.11/images/colinbjohnson/test:$tagVersion/push - neither of which seem to work. My experience with the API leads me to believe can't push both a repository (example: colinbjohnson/test) and and a tag.

It would be great to see this plugin up on the new Plugin Portal so it can be applied using the new (in Gradle 2.1+) plugins closure.

Check out:
http://plugins.gradle.org/submit

Hi,

Maybe I'm misread the documentation but I haven't found a way to use this plugin with an existing Dockerfile. It would be a great feature addition to help migrate for existing projects.

Regards,

Hi,

I'm getting the following error trying to connect to a remote docker-machine:

Docker API error: Failed to build Image:
client version 1.11 is too old. Minimum supported API version is 1.12, please upgrade your client to a newer version

Does this plugin not support version higher than 1.11? I'm using the latest version of docker.

Thanks!

is it possible to define the username and password for docker hub?

https://github.com/Transmode/gradle-docker/blob/master/src/main/groovy/se/transmode/gradle/plugins/docker/DockerPlugin.groovy#L96

In some case the user would like to create images with version different from 'latest' (the TAG field when you use 'docker images').

I created a pull request for it, but I probably made some mistake and no issue associated was created (this is my first pull request).

#5

addFile (...) and such are not very helpful when all you really want to do is add a tar.gz file into the container without expanding it. Please consider adding a copyFile method that adds a docker "COPY" statement to the Dockerfile.

Am I right that it is not possible to build and run images automatically, or am I just missing something?

BR,
wastl

Right now, docker can be run with sudo when I use the following option in build.gradle:

docker {
dockerBinary = "sudo docker"
}

This is a bit of a workaround. Some dedicated option would be nicer...

Gradle 1.x is bundled with Groovy 1.x (1.12 uses 1.8.6) and the plug-in fails to load due to a class not found error on: org/codehaus/groovy/runtime/typehandling/ShortTypeHandling. I believe that class was introduced in Groovy 2.3.

You can work around the issue by using an explicitly installed Groovy instead of the one bundled with Gradle, but this does make the plug-in harder to use (especially in environments which rely heavily on the Gradle wrapper, which ours does).

Not really an issue, just a request for info. I'm about to embark on a new project and would like to use this plug-in but version 1.3.

Any idea on when 1.3 might be officially released?

I believe I've described this problem correctly - feel free to contact if there are any questions:

Description of Problem:

When running a Docker task using an external dockerfile 'dockerfile' the context is changed from the external dockerfile's original context (for example, the root context of a project) to the context of a generated Dockerfile - typically build/docker.

Expected Function:

When using an external Dockerfile my expectation was that the context would remain relative to the original dockerfile, not a copied dockerfile. This has the impact of causing ADD and COPY file to fail, given that the context is build/docker which contains no built artifacts.

When you add a file using addFile() the plugin always copies this file across during the preparation of the staging area. However on an incremental gradle build the file might not have changed, but you will get a new file created with a newer date/time. Therefore during the build docker creates a new layer for this step.

Once this happens all layers after this added file will also not use the cache. The causes some very slow builds and is real pain, especially given the extent to which I'm using docker.

Is it possible to not copy the file across to the staging area if it hasn't changed?

This would massively improve build times.

Note:- I'm using the native docker client not the REST api.

Bit new to gradle and docker, so not sure if this is a legitimate issue, but I'm encountering the issue mentioned in the title when trying to run a task that builds a Docker image from an external dockerfile.

Currently on Ubuntu 14.04. Not sure if there's a way to run this task with elevated permissions or run docker without, just wanted to see if anyone might have a suggestion.

If multiple Docker tasks exist in the same project, they all share the same default context directory (build/docker). This leads to conflicts between the different tasks.

The workaround is to set the contextDir task property to something unique for every task. This should however be automated and the default context directory should be unique for each task. Maybe set the context directory to build/docker/${task.name}? The task name must be unique making it a good candidate for the name of the context directory.

Also: rename the property stageDir to contextDir to be more aligned with Docker lingo.

registry=http://192.168.60.41:5000

* What went wrong:
Execution failed for task ':eureka-server:buildDocker'.
> Docker execution failed
  Command line [docker build -t http://192.168.60.41:5000/eureka-server:1.0-SNAPSHOT /Users/Dreampie/Projects/DreampieProjects/cloud-server/eureka-server/build/docker] returned:
  Invalid repository name (ex: "registry.domain.tld/myrepos")

registry=192.168.60.41:5000

* What went wrong:
Execution failed for task ':eureka-server:buildDocker'.
> Docker execution failed
  Command line [docker push 192.168.60.41:5000/eureka-server:1.0-SNAPSHOT] returned:
  unable to ping registry endpoint https://192.168.60.41:5000/v0/
  v2 ping attempt failed with error: Get https://192.168.60.41:5000/v2/: EOF
   v1 ping attempt failed with error: Get https://192.168.60.41:5000/v1/_ping: EOF

docker-registry in http not https

I assumed that this method should set the repository tag to 'latest' for the image, but instead it uses the project version. I think if it set tagVersion to 'unspecified' instead of null then it would work as I assumed (of course it could be that this isn't intended, in which case you may want to consider a name change).

Exposing UDP ports on the command line or Dockerfile uses the syntax 162/udp
The exposePort(port) method expects a port to be an integer.

Thus

exposePort("162/udp")

doesn't work

This is totally user error, but I cannot figure out how to use the variant of addFile that takes a closure. My assumption is that it wants a copySpec closure, something like

{
    from 'myFile'
    into stageDir
}

But this fails on the line that uses the closure as a substitution parameter for the ADD line. There is something I'm missing here (I'm a groovy newbie).

As spring boot has its own task to create an executable jar I was wondering if this plugin will work with the bootRepackage task out of the box

It looks like the Docker push action doesn't support private (or specified) repositories:

https://github.com/Transmode/gradle-docker/blob/master/src/main/groovy/se/transmode/gradle/plugins/docker/client/NativeDockerClient.groovy#L40 shows that the push command only pushes the docker tag (with no additional options).

I would like to tag my images with a version tag and latest tag.

It appears that the code in the client's buildImage function does not account for the presence of a space in the buildDir string when doing a parameter substitution.
def cmdLine = "${binary} build -t ${tag} ${buildDir}"
This is causing a docker build failure for us in Jenkins since Jenkins uses a work directory with a space in the path because it is named after a Jenkins job, that in turn has a space in its name.

If you don't define the group property in your Gradle build, the distDocker command will not be able to actually build a docker. That is, the plugin will try to execute a command like: docker build -t /galoshe /home/vagrant/Dev/galoshe/build/docker -- note the /galoshe results in a docker error: Invalid namespace name (), only [a-z0-9_] are allowed, size between 4 and 30

Hi,
Firstly I new to using docker and have recently installed docker machine and have docker running.

I've taken a sample gradle project from https://spring.io/guides/gs/spring-boot-docker/ and I'm trying to get it to work on windows.

So I tried using the dockerBinary, but that didn't seem to work.
So thought that I would try the following

useApi true
hostUrl 'http://192.168.99.100:2376'
apiUsername 'user'
apiPassword 'password'

But when I build I get the following:

:buildDocker FAILED
FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':buildDocker'.
  java.io.IOException: Stream Closed
• Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.
  BUILD FAILED

So tried it with a stack trace and got the following:

• Exception is:
  org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':buildDocker'.
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
  at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
  at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
  at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
  at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:52)
  at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
  at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
  at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
  at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:203)
  at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:185)
  at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:62)
  at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:50)
  at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:25)
  at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:110)
  at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
  at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
  at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
  at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
  at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
  at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
  at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
  at org.gradle.initialization.DefaultGradleLauncher$4.run(DefaultGradleLauncher.java:158)
  at org.gradle.internal.Factories$1.create(Factories.java:22)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:52)
  at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:155)
  at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:36)
  at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:103)
  at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:97)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:62)
  at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:97)
  at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:102)
  at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
  at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:47)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:32)
  at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:77)
  at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:47)
  at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:51)
  at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:28)
  at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:43)
  at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:170)
  at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
  at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
  at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
  at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
  at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
  at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
  at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
  at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
  at org.gradle.launcher.Main.doAction(Main.java:33)
  at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
  at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
  at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
  at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
  Caused by: com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: Stream Closed
  at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:184)
  at com.github.dockerjava.client.utils.JsonClientFilter.handle(JsonClientFilter.java:17)
  at com.sun.jersey.api.client.filter.LoggingFilter.handle(LoggingFilter.java:217)
  at com.sun.jersey.api.client.Client.handle(Client.java:652)
  at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
  at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
  at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570)
  at com.github.dockerjava.client.command.BuildImgCmd.callDocker(BuildImgCmd.java:94)
  at com.github.dockerjava.client.command.BuildImgCmd.impl(BuildImgCmd.java:72)
  at com.github.dockerjava.client.command.BuildImgCmd.impl(BuildImgCmd.java:35)
  at com.github.dockerjava.client.command.AbstrDockerCmd.exec(AbstrDockerCmd.java:21)
  at se.transmode.gradle.plugins.docker.client.JavaDockerClient.buildImage(JavaDockerClient.java:43)
  at se.transmode.gradle.plugins.docker.client.DockerClient$buildImage.call(Unknown Source)
  at se.transmode.gradle.plugins.docker.DockerTask.build(DockerTask.groovy:244)
  at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
  at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:226)
  at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:219)
  at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:208)
  at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
  at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
  ... 57 more
  Caused by: java.io.IOException: Stream Closed
  at com.sun.jersey.core.util.ReaderWriter.writeTo(ReaderWriter.java:114)
  at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:76)
  at com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:98)
  at com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:59)
  at com.sun.jersey.api.client.RequestWriter$RequestEntityWriterImpl.writeRequestEntity(RequestWriter.java:231)
  at com.sun.jersey.client.apache4.ApacheHttpClient4Handler$2.writeTo(ApacheHttpClient4Handler.java:262)
  at org.apache.http.entity.BufferedHttpEntity.writeTo(BufferedHttpEntity.java:115)
  at org.apache.http.entity.HttpEntityWrapper.writeTo(HttpEntityWrapper.java:98)
  at org.apache.http.impl.client.EntityEnclosingRequestWrapper$EntityWrapper.writeTo(EntityEnclosingRequestWrapper.java:108)
  at org.apache.http.impl.entity.EntitySerializer.serialize(EntitySerializer.java:122)
  at org.apache.http.impl.AbstractHttpClientConnection.sendRequestEntity(AbstractHttpClientConnection.java:271)
  at org.apache.http.impl.conn.ManagedClientConnectionImpl.sendRequestEntity(ManagedClientConnectionImpl.java:197)
  at org.apache.http.protocol.HttpRequestExecutor.doSendRequest(HttpRequestExecutor.java:257)
  at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
  at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:715)
  at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:520)
  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
  at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:827)
  at com.sun.jersey.client.apache4.ApacheHttpClient4Handler.handle(ApacheHttpClient4Handler.java:170)
  ... 78 more

Any thoughts or suggestions ?

The distDocker-task in DockerPlugin.groovy will try to determine the default name of the baseImage to be used which contains java 6 / 7 or 8 according to project.targetCompatibility, but ignores any value already set in the baseImage-variable.

Since I exactly know which image I want to use, which contains my own java and environment, could the code be modified so that:

• if I specify a baseImage that name will be used, otherwise
• if no baseImage is specified (null), the code tries to determine it itself as it does now.

(and thanks for developing this task, I use it extensively!)

This leads to problems when the ADD statement refers to the directory (which doesn't exist). I think what we want to do is copy the directory itself under stage in order to preserve the expected structure.

Hi i can not build and run my spring boot project in Ubuntu server. the project uses gradle-docker.

Building 87% > :distDockerINFO[0326] POST /v1.11/build?t=callista/auth-server:latest
:distDocker FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':distDocker'.
  Docker API error: Failed to build Image:
  client is too old, minimum supported API version is 1.12, please upgrade your client to a newer version

Today:
Right now the Docker task publishes to the registry depending on a boolean which can be configured per task. This conflation of task responsibility seems to go against the Docker and Gradle way of handling artifact publishing (where there are separate tasks for building and publishing).

The Future:
I'd like to get your ideas on what you have in your minds for improving the workflow of Docker image build/publishing... I think it would be neat to wrangle the Docker publishing under the same (or similar) DSL that Gradle provides for artifact publishing (Maven/Ivy).

Whenever I make a new Docker task I pretty much always add the following:

inputs.file 'build.gradle'
outputs.dir stageDir

That way the task doesn't always execute and is re-run if I change the task definition. Seems like it might be nice to add this by default (if there is a way to only re-run if the specific task is redefined that would be great, but the build file is a reasonable proxy if that isn't possible).

Is there any way we can run the docker commands remotely over SSH, copy files with RCP? Unlike the Remote API, this does not require docker daemon to be up and running. SSH is available on any Linux/Mac system. There is aven a pure Java implementation (BSD license) at:
http://www.jcraft.com/jsch/

This is the last line of output I see

12:17:59.618 [INFO] [se.transmode.gradle.plugins.docker.DockerTask] Using the native docker binary.
> Building 95% > :server:dockerBuild

I can run the command manually with no issues. I am not sure what is going on.

There are times when using Docker as part of a build environment involves running images after they are built (for example to build an artifact that has a specialized environment). In this case it would be handy to be able to run containers as part of the build process.

Is it possible to save a docker image with this to a tar file using the save command? I couldn't find it anywhere

It's possible to build the image on my local machine but I am not able to push it to docker hub. What's the problem? Is the layer too big? It's a spring project exported as single jar-file. How can I auto-split the layer into smaller parts or reconfigure the size-limits?

Execution failed for task ':buildDocker'.
Docker execution failed
Command line [docker push name/project:latest] returned:
time="2015-05-02T13:15:46+02:00" level="fatal" msg="Failed to upload layer: Put https://cdn-registry-1.docker.io/v1/images/.../layer: write tcp 162.242.195.84:443: connection reset by peer"

The NativeDockerClient.executeAndWait() method is not handling the output buffer properly.

See: http://stackoverflow.com/questions/159148/groovy-executing-shell-commands

When the files including in the container are large (approx. 750mb), the executeAndWait method just hangs indefinitely.

The method should be written like:

private static String executeAndWait(String cmdLine) {
    print "\nexecuteAndWait(${cmdLine})\n\n"

    def process = cmdLine.execute()
    def inStream = new StringBuffer()
    def outStream = new StringBuffer()

    process.waitForProcessOutput(inStream, outStream)
    if (process.exitValue()) {
        throw new GradleException("Docker execution failed\nCommand line [${cmdLine}] returned:\n${outStream}")
    }
    return inStream
}

Env:

flozano:~$ docker --version
Docker version 1.10.3, build 20f81dd

flozano:~$ docker-machine --version
docker-machine version 0.6.0, build e27fb87

(tried with both virtualbox and vmware-fusion)

flozano:~$ gradle --version

------------------------------------------------------------
Gradle 2.12
------------------------------------------------------------

Build time:   2016-03-14 08:32:03 UTC
Build number: none
Revision:     b29fbb64ad6b068cb3f05f7e40dc670472129bc0

Groovy:       2.4.4
Ant:          Apache Ant(TM) version 1.9.3 compiled on December 23 2013
JVM:          1.8.0_72 (Oracle Corporation 25.72-b15)
OS:           Mac OS X 10.11.3 x86_64

When I run my build docker task in OS X using docker-machine I get:

[buildinfo] Not using buildInfo properties file for this build.
:myproject-docker:clean
:myproject-docker:buildDocker
      [tar] Building tar: /Users/flozano/Projects/java/myproject/myproject-docker/build/docker/add_5.tar

and the build stalls, I need to CTRL+C it or it stays like that forever. In Linux, however, it works well.

I can see the process running like:

  501 79277 79198   0  3:19PM ttys000    0:00.16 docker build -t flozano/myproject:latest /Users/flozano/Projects/java/myproject/myproject-docker/build/docker

and, actually, if I run that same command outside of docker, the build works well!

The .tar file is 21MB in the build/ dir, but inside the docker-machine instance, I can see it's only around 6MB - somehow docker fails to upload the image properly when it runs inside gradle. In fact, I can see a running "docker-tar" process inside the VM.

I have tried increasing available memory for gradle (GRADLE_OPTS="-Xmx1024m") with no luck. Any idea?

I got this error. How to fix this issue?

• Try:
  Run with --info or --debug option to get more log output.
• Exception is:
  org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':SourceInventory:distDocker'.
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:69)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:46)
  at org.gradle.api.internal.tasks.execution.PostExecutionAnalysisTaskExecuter.execute(PostExecutionAnalysisTaskExecuter.java:35)
  at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:64)
  at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
  at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:52)
  at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:52)
  at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:53)
  at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
  at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:203)
  at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:185)
  at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:62)
  at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:50)
  at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:25)
  at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:110)
  at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
  at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
  at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
  at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
  at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
  at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
  at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
  at org.gradle.initialization.DefaultGradleLauncher$4.run(DefaultGradleLauncher.java:155)
  at org.gradle.internal.Factories$1.create(Factories.java:22)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:52)
  at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:152)
  at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:33)
  at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:100)
  at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:94)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:90)
  at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:62)
  at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:94)
  at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:83)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:94)
  at org.gradle.tooling.internal.provider.runner.BuildModelActionRunner.run(BuildModelActionRunner.java:46)
  at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
  at org.gradle.tooling.internal.provider.runner.SubscribableBuildActionRunner.run(SubscribableBuildActionRunner.java:58)
  at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:43)
  at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:28)
  at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:77)
  at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:47)
  at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:52)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:37)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
  at org.gradle.util.Swapper.swap(Swapper.java:38)
  at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.health.DaemonHealthTracker.execute(DaemonHealthTracker.java:47)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:66)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:71)
  at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.health.HintGCAfterBuild.execute(HintGCAfterBuild.java:41)
  at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
  at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
  at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:246)
  at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
  at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
  Caused by: org.gradle.api.UncheckedIOException: java.io.IOException: Cannot run program "docker": error=2, No such file or directory
  at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:43)
  at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:78)
  at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.doExecute(AnnotationProcessingTaskFactory.java:226)
  at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:219)
  at org.gradle.api.internal.project.taskfactory.AnnotationProcessingTaskFactory$StandardTaskAction.execute(AnnotationProcessingTaskFactory.java:208)
  at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:585)
  at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:568)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:80)
  at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:61)
  ... 70 more
  Caused by: java.io.IOException: Cannot run program "docker": error=2, No such file or directory
  at se.transmode.gradle.plugins.docker.client.NativeDockerClient.executeAndWait(NativeDockerClient.groovy:45)
  at se.transmode.gradle.plugins.docker.client.NativeDockerClient.buildImage(NativeDockerClient.groovy:34)
  at se.transmode.gradle.plugins.docker.client.DockerClient$buildImage.call(Unknown Source)
  at se.transmode.gradle.plugins.docker.DockerTask.build(DockerTask.groovy:244)
  at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:75)
  ... 77 more
  Caused by: java.io.IOException: error=2, No such file or directory
  ... 82 more

The ADD instruction also supports fetching files from a remote URL. However the plugin's addFile method does not allow this.

A workaround is to call the generic addInstruction like so:

task.addInstruction('ADD', '<URL>')

The addFile method should support remote files.

The example you gave in the "Standalone" section in your readme implies that the gradle-docker plugin can be found on Maven Central. That example fails because the plugin cannot be found.

Here is the error I get:

    > Could not resolve all dependencies for configuration ':classpath'.
       > Could not find se.transmode.gradle:gradle-docker:1.0.

For Spring Boot applications, most builds don't change dependencies, so the only file that actually changes is the ${applicationName}.jar file. However, the Dockerfile produced by this plugin ADDs the entire distribution as a single Docker command. So even a 1 byte change will potentially cause a new 40+ MB layer to be created by docker.

If the Dockerfile instead separated the "semi-static" dependencies from the frequently changing files as separate ADD/COPY steps, then the size of the docker image layers created would be substantially reduced.

I've prototyped this locally outside of this plugin via:

task unzipDistWithoutBootJar(type: Copy, dependsOn: distZip) {
    from zipTree(file("${buildDir}/distributions/${applicationName}.zip"))
    into "${buildDir}/docker"
    exclude "**/${applicationName}.jar"
}

task unzipBootJar(type: Copy, dependsOn: [unzipDistWithoutBootJar]) {
        from zipTree("${buildDir}/distributions/${applicationName}.zip")
        into "${buildDir}/docker"
        includeEmptyDirs = false
        include "**/${applicationName}.jar"
        doLast {
                def file = new File("${buildDir}/docker/${applicationName}/lib/${applicationName}.jar")
                def dir = new File("${buildDir}/docker")
                file.renameTo(new File(dir, file.getName()))
        }
}

task createDockerfile(dependsOn: distZip) {
    doLast {
        new File("${buildDir}/docker/Dockerfile").text = """
FROM ubuntu
EXPOSE 8080

COPY ${applicationName} /${applicationName}/
COPY ${applicationName}.jar /${applicationName}/lib/${applicationName}.jar

ENTRYPOINT ["/${applicationName}/bin/${applicationName}"]
"""
    }
}

task distDocker2(type: Exec, dependsOn: [build, unzipDistWithoutBootJar, unzipBootJar, createDockerfile]) {
    workingDir "${buildDir}/docker"
    commandLine 'docker', 'build', '-t', "${project.group}/${applicationName}", '.'
}

Some posts I've seen mention that you also need to force the timestamp of the "semi-static" resources to be the same. However in my testing I didn't find that to be necessary. Changing just timestamps on the files still resulted in the semi-static layer being cached. Only changing the contents of a file or adding/removing files seemed to cause a new layer.

The end result is a substantial savings in docker image layer size. The key benefit, at least for my current use case, is that CI builds can now push new docker images on every build without causing undo burden on our docker repository storage.

I'll see if I can incorporate this into a fork, but wanted to get the issue out there for a future roadmap. My thought is to make it a configurable property, something like "skinnyLayers: true", and possibly also have some way to make the description of what is "semi-static" versus what is "dynamic" configurable.