googlecloudplatform / golang-docker Goto Github PK

Docker images for Go on Google App Engine

License: Apache License 2.0

Shell 51.79% Go 39.53% Dockerfile 8.68%

golang-docker's Introduction

golang-docker

This repository contains sources for the Go App Engine Flex build pipeline. This build pipeline utilizes Google Cloud Build to produce the application image. The go-1.x.yaml configuration files are used in the build pipeline. User can now specify a Go version in app.yaml via runtime: go1.x, where 1.x can be 1.8, 1.9, etc. The resulting application image uses gcr.io/distroless/base as the base image, which contains a minimal Linux, glibc-based system.

The builder image gcr.io/gcp-runtimes/go1-builder is tagged with supported Go major versions for builds to be able to target specific Go version to build against. Run the following gcloud command to list supported versions --

$ gcloud container images list-tags gcr.io/gcp-runtimes/go1-builder

Use only the Go major version tags e.g. 1.8, 1.9, instead of the unique version-datetime tags.

There are different ways to reuse the go1-builder image outside of AppEngine Flex. Here are a couple of examples.

Multi-stage Dockerfile

Read up on how to use Docker's multi-stage builds.

Following is a trivial example of a Dockerfile with multi-stage build using the go1-builder image to bring in the Go SDK. The Go SDK is located under /usr/local/go of the builder image.

FROM gcr.io/gcp-runtimes/go1-builder:1.9 as builder

WORKDIR /go/src/app
COPY main.go .

RUN /usr/local/go/bin/go build -o app .

# Application image.
FROM gcr.io/distroless/base:latest

COPY --from=builder /go/src/app/app /usr/local/bin/app

CMD ["/usr/local/bin/app"]

Google Cloud Build

Read up on Google Cloud Build.

You can use one of the go-1.x.yaml files for configuration file. The builder image's ENTRYPOINT is the go-build.sh script, which expects a certain layout of the uploaded directory. In particular, it expects --

_gopath/src directory containing Go source files within package directory structure.
_gopath/main-package-path file containing the import path of the main application to be built.
top level directory can contain resource files that will be bundled into the application image's /app directory.

The build script will build the application as referenced in _gopath/main-package-path file and produce an application Dockerfile which the next step will run a docker build on.

You can customize your own cloudbuild.yaml file with your own Dockerfile.

golang-docker's People

Contributors

Stargazers

Watchers

golang-docker's Issues

image does not appear to support vendoring

I am using the app engine flex environment.

When I deploy, the cloud build fails:

Step 3/3 : RUN go-wrapper install -tags appenginevm
---> Running in e3aebcd0fd0e
exec go install -v -tags appenginevm
main.go:10:2: cannot find package "github.com/gorilla/mux" in any of:
/usr/local/go/src/github.com/gorilla/mux (from $GOROOT)
/go/src/github.com/gorilla/mux (from $GOPATH)
/go/src/app/_gopath/src/github.com/gorilla/mux
The command '/bin/sh -c go-wrapper install -tags appenginevm' returned a non-zero code: 1
ERROR
ERROR: build step "gcr.io/cloud-builders/docker@sha256:c99ef71791f407f8d2deaf782feceb00f4c5b59d0e8f7e50160acf9dd638d6failed: exit status 1

I checked the archive used to build the container. The code in question is in:

_gopath/src/github.com/btc/apartment-service/vendor/github.com/gorilla/mux

apartment-service (master) λ. gcloud  version                                                                                                                                                                  2
Google Cloud SDK 179.0.0
app-engine-go
app-engine-python 1.9.62
beta 2017.09.15
bq 2.0.27
core 2017.11.06
gcloud
gsutil 4.28
kubectl

Tagging versions?

Is there a way to see what versions are available? It looks like docs offer images without any tags.

When I pulled gcr.io/google-appengine/golang I observed that it is go1.6.3, but go1.8 is now available. Is it because this image is designed for App Engine or is there some other reason?

add :stable label for base

That always point to the :stable version (i.e: 1.2.2).

stackdriver debug broke hello world

steps to reproduce:

checkout 54e58ac
docker build -t golang ./base
clone helloworld
add a Dockerfile:

FROM golang

COPY . /go/src/app
RUN go-wrapper install -tags appenginevm

change runtime to custom in app.yaml
aedeploy docker build -t helloworld
docker run --rm helloworld:

+ which app
+ exec go-cloud-debug -sourcecontext ./source-context.json -appmodule default -appversion default -- /go/bin/app
2016/12/30 17:02:02 Debuglet initialization: metadata: GCE metadata "project/numeric-project-id" not defined
2016/12/30 17:02:02 Debuglet initialization: metadata: GCE metadata "project/project-id" not defined
2016/12/30 17:02:02 Reading source context file: reading file "./source-context.json": open ./source-context.json: no such file or directory
2016/12/30 17:02:02 Error connecting to Cloud Debugger: google.DefaultTokenSource: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

If I do this one commit before current head (i.e. e3fa19c) it works without problems.

I get the same error when I actually deploy to ae using aedeploy gcloud app deploy.

Is Google abandoning App Engine Flexible?

No updates for over 2 years. Issues and pull requests ignored. There are known vulnerabilities to Go 1.15, so it's becoming a security issue.

Google should clearly communicate if App Engine Flexible is being abandoned.

Add Go 1.16

Add SHA checking

Instead of curl ... | tar, curl then verify the SHA256

Something like this... (from GoogleCloudPlatform/appengine-sidecars-docker#21)

GO_ARCHIVE="https://storage.googleapis.com/golang/go${GO_VERSION}.linux-amd64.tar.gz"
GO_ARCHIVE_SHA="${GO_ARCHIVE}.sha256"

curl -fsSL "$GO_ARCHIVE" -o go.tar.gz
curl -fsSL "$GO_ARCHIVE_SHA" -o go.tar.gz.sha256

echo "$(cat go.tar.gz.sha256)  go.tar.gz" | sha256sum -c -

update readme with vendoring details

Not a bug as such, it would be nice to add a note to the readme on how to get the go app that is vendored built using a multistage Dockerfile.

The current readme

WORKDIR /go/src/app
COPY main.go .

RUN /usr/local/go/bin/go build -o app .

doesn't seem to work as /go/src/app is not it gopath. Happy to submit a pr with updated readme.

WORKDIR /root/go/src/app
COPY . .

RUN /usr/local/go/bin/go build -o app .

setting the workdir to /root/go/src/app did the magic for me :-) . I guess updating the WORKDIR to /root/go/src/app works for everyone including apps coming with vendor dependencies.

Documentation

Hi,
I'm trying to deploy this but it's too much confusing to me since I'm kinda young to the gcloud and docker.
Do you guys think it would make sense to have a step-by-step guide on how to deploy and execute on the gcloud?

Thanks.

Add end-to-end tests

We should add some end-to-end tests.

Build the base image
Use it to build an image with some predefined code
Run the code and check the behaviour/output.

Document how to run this image on GKE

add multiple compile target for google/golang

The default should be:
google/golang:1.2.2.linux-amd64

But we could also provide other flavor for cross compilation purpose, ex:
google/golang:1.2.2.darwin-amd64

Or even one with all target built from sources:
google/golang:1.2.2.all

We could switch to building from sources as there should be little to no overhead in the image size (we already need build-essentials for cgo).

Use Go 1.5.1.

vendoring is broken

For package vendoring dependencies, (like coreos/etcd) nested dependencies will never get resolved locally, as the import path is app, instead of the fully qualified one.

A proper solution is to allow package who vendor dependencies to specify their original import path with a .godir text file or through Godeps, and default on app only if the fully qualified import path cannot be infered.

go-wrapper needs to be copied into this repo

The go-wrapper from the official Docker golang base image needs to be copied into this repo:
https://github.com/docker-library/golang/blob/f1f65c0ab0097a5e3d079d5a74e2468e8d47563d/1.5/go-wrapper

I will investigate how to do this.

Build gin-gonic go-module based website fails ( permission denied )

Hi there, i've tried multiple solutions and none of them helped. Can not build my project:

# Dockerfile (builder)
FROM gcr.io/gcp-runtimes/go1-builder:1.14 as builder
WORKDIR /root/go/src/app/src/github.com/AndriiAndrus/GoWebsite
COPY . .
RUN GO111MODULE=on GOOS=linux GOARCH=amd64 /usr/local/go/bin/go build -o app .

Fails with multiple "permission denied":

/root/go/pkg/mod/github.com/gorilla/[email protected]/securecookie.go:21:2: open /usr/local/go/src/strconv/atob.go: permission denied
/root/go/pkg/mod/github.com/disintegration/[email protected]/io.go:15:2: open /usr/local/go/src/strings/builder.go: permission denied
/root/go/pkg/mod/github.com/disintegration/[email protected]/histogram.go:5:2: open /usr/local/go/src/sync/cond.go: permission denied
/root/go/pkg/mod/go.uber.org/[email protected]/atomic.go:27:2: open /usr/local/go/src/sync/atomic/asm.s: permission denied
/root/go/pkg/mod/gopkg.in/natefinch/[email protected]/chown_linux.go:5:2: open /usr/local/go/src/syscall/asm9_unix1_amd64.s: permission denied
/root/go/pkg/mod/github.com/go-ozzo/ozzo-validation/[email protected]/error.go:13:2: open /usr/local/go/src/text/template/doc.go: permission denied
/root/go/pkg/mod/github.com/gorilla/[email protected]/securecookie.go:23:2: open /usr/local/go/src/time/example_test.go: permission denied
/root/go/pkg/mod/github.com/asaskevich/[email protected]/utils.go:11:2: open /usr/local/go/src/unicode/casetables.go: permission denied
/root/go/pkg/mod/google.golang.org/[email protected]/internal/encoding/text/decode_string.go:12:2: open /usr/local/go/src/unicode/utf16/export_test.go: permission denied
/root/go/pkg/mod/golang.org/x/[email protected]/html/escape.go:10:2: open /usr/local/go/src/unicode/utf8/example_test.go: permission denied
/root/go/pkg/mod/github.com/gin-gonic/[email protected]/internal/bytesconv/bytesconv.go:5:2: open /usr/local/go/src/unsafe/unsafe.go: permission denied
The command '/bin/sh -c GO111MODULE=on GOOS=linux GOARCH=amd64 /usr/local/go/bin/go build -o app .' returned a non-zero code: 1

Please help <3

error: cannot run ssh: No such file or directory

This is what I get when I try to build the Docker (on AWS Elastic Beanstalk). It fails to get the dependencies from Github. Any ideas?

Support passing arguments to the running process

Arguments don't get passed over to the executable when using only /bin/go-run.

Document built-in stackdriver integration

We should document any built-in strackdriver integration in this image. I believe it enables the debugger by default.

base: need appenginevm build tag

The current base image doesn't set the appenginevm build tag.

Alpine-based image?

Is there a plan to offer an alpine-based (smaller) alternative for this base image, like docker.io/library/golang:1.8-alpine?

Running docker images locally (cont'd #60)

Opening a new issue to keep these clean and self contained. Continuing from #60 : how do you run a custom built docker image locally? Now that it uses go-cloud-debug and source-context.json, which is not created when I do docker run myapp, I can't run the image outside the app engine. Which was a pretty dandy feature to have.

( @shantuo )

custom project packages not located

Packages that are go get-able work fine, but custom packages that reside within the project cannot be located

Error: Missing Git

Relevant lines from build output:

Building server
Step 1/7 : FROM gcr.io/google-appengine/golang:1.8
1.8: Pulling from google-appengine/golang
Digest: sha256:21a08fb893abcca05d8c39b71d10c56ab9a13120b86f61d498c986a1de6b5cd1
Status: Downloaded newer image for gcr.io/google-appengine/golang:1.8
 ---> dd65085f4df4
Step 2/7 : WORKDIR /go/src/app
 ---> 84977f9774c0
Removing intermediate container 9c757579198e
Step 3/7 : COPY . .
 ---> cb7ed7afc0a4
Step 4/7 : RUN go-wrapper download
 ---> Running in 2743dd18164c
+ exec go get -v -d
github.com/go-sql-driver/mysql (download)
go: missing Git command. See https://golang.org/s/gogetcmd

my Dockerfile

FROM gcr.io/google-appengine/golang:1.8

WORKDIR /go/src/app
COPY . .

RUN go-wrapper download
RUN go-wrapper install

CMD ["go-wrapper", "run"]

I just switched to this base image and ran into this issue. Previously, I was using golang:1.8 base image and everything worked fine. a side Q: what's the difference b/w the gcr.io golang image and the official golang:1.8?

Investigate image size

Our images are very large for a variety of reasons.

$ docker history gcr.io/google_appengine/golang:2016-08-10_18_45
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
e06997a214ba        31 hours ago        /bin/sh -c #(nop) CMD ["go-wrapper" "run"]      0 B
37fd2f472cbc        31 hours ago        /bin/sh -c #(nop) WORKDIR /go/src/app           0 B
cfbc911eea0e        31 hours ago        /bin/sh -c ln -s /go/src/app /app               11 B
0aabc89a0aa9        31 hours ago        /bin/sh -c curl https://raw.githubusercontent   2.481 kB
94e29f3289f3        31 hours ago        /bin/sh -c mkdir -p /go/src/app /go/bin && ch   0 B
d2cfa290d49e        31 hours ago        /bin/sh -c #(nop) ENV GOPATH=/go:/go/src/app/   0 B
886366f8a001        31 hours ago        /bin/sh -c #(nop) ENV PATH=/go/bin:/usr/local   0 B
f53f4970bde3        31 hours ago        /bin/sh -c curl -sSL https://golang.org/dl/go   318.1 MB
45b58d51fac7        31 hours ago        /bin/sh -c #(nop) ENV GO_WRAPPER_COMMIT=3cdd8   0 B
05b26cbd4669        31 hours ago        /bin/sh -c #(nop) ENV GO_VERSION=1.6.3          0 B
93a107a7448c        31 hours ago        /bin/sh -c apt-get update     && apt-get inst   157.5 MB
5d735d16c1df        3 weeks ago         /bin/sh -c apt-get -q update &&     apt-get i   7.911 MB
193fa0359d01        3 weeks ago         /bin/sh -c #(nop) ENV PORT=8080                 0 B
e8eb529bd074        3 weeks ago         /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninte   0 B
abf0692dab5b        3 weeks ago         /bin/sh -c #(nop) CMD []                        0 B
b21b3e3bc691        8 weeks ago         /bin/sh -c #(nop) ADD file:8086d9d18e7f7c09e5   123.6 MB
b29ae6f49e5a        8 weeks ago         /bin/sh -c #(nop) ARG DEBIAN_SUITE              0 B
3bc7cdb36864        12 months ago       /bin/sh -c #(nop) ENV PATH=/usr/local/sbin:/u   0 B
4bfc20f0ed66        12 months ago       /bin/sh -c #(nop) ENV DEBIAN_FRONTEND=noninte   0 B

Please update to Go 1.6.3

Go 1.6.1 includes some security fixes.
https://groups.google.com/forum/#!msg/golang-announce/9eqIHqaWvck/kXsfO0ogLAAJ

GO_WRAPPER_COMMIT should to be updated to d7e2a8d90a9b8f5dfd5bcd428e0c33b68c40cc19
https://github.com/docker-library/golang/commits/master

Thanks.

P.S. The CLA links go to code.google.com which redirect to https://cla.developers.google.com. May want to update those links.