prometheus / golang-builder Goto Github PK

Prometheus Golang builder Docker images

License: Apache License 2.0

Makefile 11.49% Shell 41.09% Dockerfile 22.62% Go 24.80%

golang-builder's Introduction

Prometheus

Visit prometheus.io for the full documentation, examples and guides.

Prometheus, a Cloud Native Computing Foundation project, is a systems and service monitoring system. It collects metrics from configured targets at given intervals, evaluates rule expressions, displays the results, and can trigger alerts when specified conditions are observed.

The features that distinguish Prometheus from other metrics and monitoring systems are:

A multi-dimensional data model (time series defined by metric name and set of key/value dimensions)
PromQL, a powerful and flexible query language to leverage this dimensionality
No dependency on distributed storage; single server nodes are autonomous
An HTTP pull model for time series collection
Pushing time series is supported via an intermediary gateway for batch jobs
Targets are discovered via service discovery or static configuration
Multiple modes of graphing and dashboarding support
Support for hierarchical and horizontal federation

Architecture overview

Install

There are various ways of installing Prometheus.

Precompiled binaries

Precompiled binaries for released versions are available in the download section on prometheus.io. Using the latest production release binary is the recommended way of installing Prometheus. See the Installing chapter in the documentation for all the details.

Docker images

Docker images are available on Quay.io or Docker Hub.

You can launch a Prometheus container for trying it out with

docker run --name prometheus -d -p 127.0.0.1:9090:9090 prom/prometheus

Prometheus will now be reachable at http://localhost:9090/.

Building from source

To build Prometheus from source code, You need:

Start by cloning the repository:

git clone https://github.com/prometheus/prometheus.git
cd prometheus

You can use the go tool to build and install the prometheus and promtool binaries into your GOPATH:

GO111MODULE=on go install github.com/prometheus/prometheus/cmd/...
prometheus --config.file=your_config.yml

However, when using go install to build Prometheus, Prometheus will expect to be able to read its web assets from local filesystem directories under web/ui/static and web/ui/templates. In order for these assets to be found, you will have to run Prometheus from the root of the cloned repository. Note also that these directories do not include the React UI unless it has been built explicitly using make assets or make build.

An example of the above configuration file can be found here.

You can also build using make build, which will compile in the web assets so that Prometheus can be run from anywhere:

make build
./prometheus --config.file=your_config.yml

The Makefile provides several targets:

build: build the prometheus and promtool binaries (includes building and compiling in web assets)
test: run the tests
test-short: run the short tests
format: format the source code
vet: check the source code for common errors
assets: build the React UI

Service discovery plugins

Prometheus is bundled with many service discovery plugins. When building Prometheus from source, you can edit the plugins.yml file to disable some service discoveries. The file is a yaml-formated list of go import path that will be built into the Prometheus binary.

After you have changed the file, you need to run make build again.

If you are using another method to compile Prometheus, make plugins will generate the plugins file accordingly.

If you add out-of-tree plugins, which we do not endorse at the moment, additional steps might be needed to adjust the go.mod and go.sum files. As always, be extra careful when loading third party code.

Building the Docker image

The make docker target is designed for use in our CI system. You can build a docker image locally with the following commands:

make promu
promu crossbuild -p linux/amd64
make npm_licenses
make common-docker-amd64

Using Prometheus as a Go Library

Remote Write

We are publishing our Remote Write protobuf independently at buf.build.

You can use that as a library:

go get buf.build/gen/go/prometheus/prometheus/protocolbuffers/go@latest

This is experimental.

Prometheus code base

In order to comply with go mod rules, Prometheus release number do not exactly match Go module releases. For the Prometheus v2.y.z releases, we are publishing equivalent v0.y.z tags.

Therefore, a user that would want to use Prometheus v2.35.0 as a library could do:

go get github.com/prometheus/[email protected]

This solution makes it clear that we might break our internal Go APIs between minor user-facing releases, as breaking changes are allowed in major version zero.

React UI Development

For more information on building, running, and developing on the React-based UI, see the React app's README.md.

More information

Godoc documentation is available via pkg.go.dev. Due to peculiarities of Go Modules, v2.x.y will be displayed as v0.x.y.
See the Community page for how to reach the Prometheus developers and users on various communication channels.

Contributing

Refer to CONTRIBUTING.md

License

Apache License 2.0, see LICENSE.

golang-builder's People

Contributors

Stargazers

Watchers

golang-builder's Issues

issue with 1.17

go: downloading github.com/pkg/errors v0.8.1
2021/08/31 12:22:01 processing 1.16
2021/08/31 12:22:01 no version change for Go 1.16.7
2021/08/31 12:22:01 processing 1.17
2021/08/31 12:22:01 failed to detect current version of 1.17: couldn't get exact version for 1.17
exit status 1

Consider renaming latest minor version images

Golang uses $major.$minor version names for patch version 0. Golang builder uses the same naming scheme for the "latest patch version" image. Given an image name golang-1.6 it's not clear to a go developer whether that means 1.6.3 or 1.6.0.

What do you think about using 1.6.x as image name for "the latest patch version"?

Increment go versions for CVE-2023-45288

Fixing this CVE requires incrementing go versions to 1.21.9 and 1.22.2.

Is this likely to be done soon?

Cannot crossbuild with promu with latest golang-builder images

After this change to the golang-builder base image 1.18: https://quay.io/repository/prometheus/golang-builder?tab=history&tag=latest on June 8, any crossbuilds with promu that utilize the golang-builder image is completely broken. The container cannot start:

build
bash build_ui.sh --all
build module/lezer-promql
make: *** [Makefile:48: ui-build] Error 243
!! The base builder docker image exited unexpectedly: exit status 2

This is on debian linux. I ran promu crossbuild -p windows/amd64 as well as linux/amd64, both failed with same error.

Tests fail for pull requests from forked repositories

Reported from #51 (comment)

The CI failed because OSXCROSS_SDK_URL isn't set (https://circleci.com/gh/prometheus/golang-builder/165). This relates probably to the migration from Circle CI 1.0 to 2.0 and the fact that by default, builds from forked pull requests can't access secrets (see https://circleci.com/docs/2.0/oss/#pass-secrets-to-builds-from-forked-pull-requests).

Not sure what is the fix for this problem.

debian:buster instead of :stable to allow for s390x builds

From https://groups.google.com/forum/#!msg/prometheus-developers/ZjWZR1kVMbg/Ym2fa43iBgAJ

They'd like to contribute building binaries for s390x but they require debian:buster because it provides crossbuild-essentials-s390x.

Thoughts on moving to :buster @SuperQ @sdurrheimer @grobie?

Run additional test targets

Currently, passing -T leads to the test make target being run.

It would be handy if it were possible to run the other testing-related targets (check_license, style unused, and staticcheck).

Assuming there is no objection to this, I'm happy to put up a PR. I could:

just keep a single -T flag, and expand the set of make targets that it runs to include the above
add some separate testing flag(s) to cover those other targets

Any preferences?

Add some documentation

I spent some time today figuring out how all this is actually working. I think we should add just a few words to the README explaining what the builder images contain, how they get build and how we use them to cross compile. Just a few sentences would be enough.

Update MAINTAINERS.md

Seems like the maintainer on this repo is old, do we plan on updating the file?

Is there a maintainer at the moment?

Upgrade base to Debian bookworm

Debian 12 / bookworm is expected sometime in 2023. There are some TODO items.

riscv64

Add packages:

libasan6-riscv64-cross
libitm1-riscv64-cross
libubsan1-riscv64-cross

Use json to fetch latest versions

Go provides a json url to fetch the latest versions: https://go.dev/dl/?mode=json

We should use this data rather than polling for file download URLs.

Remove the entrypoint from the interface

I'd suggest to not set an entrypoint in the Dockerfile but leave that to the caller. The builder.sh script should be in the PATH so that one can run the docker container with docker run --rm prom/golang-builder:1.5.3-main builder.sh [options]. This will also allow to provide more than one command later, for diagnosis for example.

Does this support PowerPC 32bit？

I'm trying to compile a go program to run on PowerPC (made by freescale,running in 32bit,big endian). As I can see in your guide, the powerpc environment you use is ppc 64bit little endian.
Does your docker image support PowerPC 32bit big endian?

Add Go 1.10.4

Supersedes 1.10.3.

Document make interface

As I understand, the golang builder requires a Makefile providing build target. A set of environment variables will be available and there are expectations around the file location of the build output produced by the make target.

It'd be great to document this interface in the README.

Minor version upgrades don't update the `main` Dockerfile

It seems like there is a bug in the updater tool that misses updating the version of the base FROM in the main Dockerfile.

See: #234

Add Go 1.11

To use the new user annotation API of the runtime/trace package (see prometheus/prometheus#4568 (comment)). And also to experiment with modules.

I'll send a PR soon.

`gotestsum` not present in 1.17

Starting with the PR that introduced the 1.17 builder image into prometheus/prometheus, test results haven't been uploaded, instead rendering this error:

Unable to save test results from /root/project/test-results
Error error accessing path "/root/project/test-results": stat /root/project/test-results: no such file or directory
Found no test results, skipping

As can be seen from this run about a week earlier, this error didn't appear with the 1.16 builder image.

This is because gotestsum is somehow no longer present in 1.17.

>> running all tests
GO111MODULE=on gotestsum --junitfile test-results/unit-tests.xml -- -race -p 2 ./...

(1.16)

>> running all tests
GO111MODULE=on go test -race -p 2 ./...

(1.17)

https://github.com/prometheus/prometheus/blob/c564984daa96a9f49ab6dc3c3db9b29b580cae38/Makefile.common#L72-L79

Deduplicate builder.sh

Not sure I fully understand the intend, but we have builder.sh in both the base image as well as in main and the platform specific variants. Since the script has platform specific conditions I'm wondering whether we can just have one in the base image? Or what is the reason to have one per arch?

/cc @sdurrheimer