Currently we override env with every tool version. Maybe a better alternative is to use ubuntu update-alternatives.

https://serverfault.com/questions/631447/how-to-use-update-alternatives-per-user/811377#811377

Signing generated containers allows end users to verify that any container they pull actually came from us. With GitHub Actions new support for OIDC ¹ it's now possible to use cosign ² to sign containers in GitHub Actions without any human interaction or storing long-lived keys as secrets.

Here's a demo someone put together explaining how to set it all up: https://github.com/chrisns/cosign-keyless-demo

• blocked by renovatebot/renovate#12820
• helps for renovatebot/renovate#6049

• extract prepare-tool
• allow user install

install-buildpack should check if there are any *.crt files in /usr/local/share/ca-certificates/ and if so then run update-ca-certificates automatically.

We should use copy all test repositories into this org so that we don't depend on @renovate-testing's repos.

I'm thinking to use a prefix convention like testing-, e.g. github.com/containerbase/testing-bundler, etc.

Add a prepare-tool to install all required distro base dependencies (apt).

Ubuntu base images are only updated periodically, and may have vulnerabilities in packages which are fixed in the apt repository.

How can we balance such upgrading while also retaining some caching ability?

Example:

ubuntu:20.04 (ubuntu 20.04)
===========================
Total: 2 (UNKNOWN: 0, LOW: 1, MEDIUM: 1, HIGH: 0, CRITICAL: 0)

+-------------+------------------+----------+-------------------+------------------+
|   LIBRARY   | VULNERABILITY ID | SEVERITY | INSTALLED VERSION |  FIXED VERSION   |
+-------------+------------------+----------+-------------------+------------------+
| libgcrypt20 | CVE-2021-40528   | MEDIUM   | 1.8.5-5ubuntu1    | 1.8.5-5ubuntu1.1 |
+             +------------------+----------+                   +                  +
|             | CVE-2021-33560   | LOW      |                   |                  |
+-------------+------------------+----------+-------------------+------------------+

Meanwhile the same image with RUN apt-get update && apt-get -y upgrade && rm -rf /var/lib/apt/lists/* results in no vulnerabilities.

Ref: https://pythonspeed.com/articles/security-updates-in-docker/

At shell check to ci. Use husky and lint-staged for pre-commit git hook to validate changed files

/Cc @Chumper

• major as feat with breaking change

refs:

• #168
• renovatebot/renovate#12509

If custom certificates are provided, we should automatically configure Node.js to use them. By default they use their own bundled CA certs, so I think would ignore any custom certs added to Linux.

Should we configure --use-openssl-ca or SSL_CERT_DIR or SSL_CERT_FILE?

Hi folks,

We are using renovate with some Bazel repositories and it would be really helpful to install https://github.com/bazelbuild/bazelisk onto this default docker image.

I want to gauge your thought before putting our a PR

The installation will looks roughly like this but im not quite sure how install-tool works

RUN curl -fLo /usr/local/bin/bazel https://github.com/bazelbuild/bazelisk/releases/download/v1.6.1/bazelisk-linux-amd64 && \
    chown root:root /usr/local/bin/bazel && \
    chmod 0755 /usr/local/bin/bazel

renovatebot/docker-buildpack#51 (comment)

We should support a IGNORED_TOOLS build argument, which will be a comma-separated list. If the list is non-empty, any tool within should be skipped for install-tool.

Example: IGNORED_TOOLS=PIPENV,HELM

Maybe we can make it case-insensitive? e.g. allowing IGNORED_TOOLS=pipenv,helm too.

Currently these are both installed using install-gem

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Location: renovate.json
Error type: Invalid JSON (parsing failed)
Message: Syntax error: expecting end of expression or separator near "node

JRE only exists for LTS releases

• java should first try jre and fallback to jdk
• java-jre should install jre and fail if missing
• java-jdk should always install jdk and fail if missing

This maybe affects anyone building with buildpack? :(

renovatebot/docker-buildpack#163 (comment)

Missing parts:

• add shell wrapper for install-pip
• add shell wrapper for install-gem
• java
• erlang
• docker

All of our tool install scripts (except nix 😉) use the #!/bin/bash shebang, but these aren't portable, and aren't the best practice:tm:. Changing these to #!/usr/bin/env bash is a simple fix.

We should also cosign this image

/cc @JamieMagee

Make all possible tools runtime installable.

TODO

• #121
• #197
• #213
• bundler #251
• cocoapods #251
• composer
• docker #651
• #118
• elixir #376
• #318
• #200
• git-lfs #652
• #653
• #119
• gradle
• hashin (pip_requirements) #639
• #199
• java
• #178
• nix #573
• #217
• npm
• pipenv #639
• pip-tools (pip-compile) #639
• #120
• pnpm
• poetry
• #633
• #404
• #475
• #571
• sbt
• scala
• #576
• #181, #198
• yarn

Reproduction:

❯ docker run --rm renovate/renovate bash -c "install-tool npm 6.14.15 || true && echo 'Installed npm version is:' && npm --version" 
Installing tool npm v6.14.15
/home/ubuntu/npm/6.14.15/bin/npm -> /home/ubuntu/npm/6.14.15/lib/node_modules/npm/bin/npm-cli.js
/home/ubuntu/npm/6.14.15/bin/npx -> /home/ubuntu/npm/6.14.15/lib/node_modules/npm/bin/npx-cli.js
+ [email protected]
added 437 packages from 891 contributors in 20.191s
npm ERR! It doesn't look like npm is installed.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/ubuntu/.npm/_logs/2021-12-08T09_01_28_815Z-debug.log
Installed npm version is:
6.14.15

It would be useful if we can write the installed version to e.g. /home/user/npm/version, /home/user/node/version, etc (or wherever is most appropriate). That way an application like Renovate can look for a such a file to know in advance what's been installed.

• download linux-amd64 from releases
• extract
• copy binary to path /usr/local/bin (alternate is using the included install.sh)
• run git lfs install as ubuntu user

blocks: renovatebot/renovate#6842

We should use other install method, as its faster (install) that the apt version. We should also use this because the available version on ubuntu is old.

Ref renovatebot/docker-buildpack#139

We use the renovate/renovate docker image. Recently that image started to fail to start in CircleCI due to:

CircleCI was unable to start the container because of a userns remapping failure in Docker.

This typically means the image contains files with UID/GID values that are higher than Docker and CircleCI can use.

Checkout our docs https://circleci.com/docs/2.0/high-uid-error/ to learn how to fix this problem.
Original error: CircleCI was unable to start the container because of a userns remapping failure in Docker.

This typically means the image contains files with UID/GID values that are higher than Docker and CircleCI can use.

Checkout our docs https://circleci.com/docs/2.0/high-uid-error/ to learn how to fix this problem.
Original error: failed to register layer: Error processing tar file(exit status 1): Container ID 1407182137 cannot be mapped to a host ID

Following the instructions on that referenced page in error, I did:

$ docker run --rm -it --entrypoint bash renovate/renovate

(in container): $ find / \( -uid 1407182137 \)  -ls 2>/dev/null
  8000928      8 -rw-r--r--   1 1407182137 1059092950     7444 Aug 19 16:55 /usr/local/poetry/1.1.8/lib/poetry/_vendor/py2.7/backports/entry_points_selectable.py

And this file appears to be installed from install-tools poetry in this repo. I hope this is the right place for this issue.

As part of renovatebot/renovate#12720, I need to add support for jsonnet-bundler in buildpack.

Jsonnet-bundler can be installed in two ways, neither requires root.

1. go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/[email protected]
2. github release binary

An even more advanced approach here would be that we create our own tools directory in $PATH, e.g. containing node, poetry, gradle, etc. But each is in fact a wrapper/stub and its purpose is to dynamically install the required tool + language and then pass the same parameters to the tool itself after it's installed.

e.g. the application like Renovate will simply call poetry via child process, which then kicks off:

• Our own custom shell script
• Installs python if missing (According to PYTHON_VERSION)
• Installs poetry
• Calls newly installed poetry and passes all args to it

Depends on

• #16
• #771
• #1384
• #1385

For locking some pip lockfiles additional dependencies are required

• default-libmysqlclient-dev
• build-essential ❓ 👉 gcc
• libpq-dev #30
• renovatebot/renovate#10850

blocked by

• #121

• #177 Allow installing multiple dotnet versions to same folder (like official installer does)
• Allow installation without root
• Create prepare tool to install base deps which need root
• blocked by #121

ref #16

We should support an optional cache dir /volume to cache downloads, so we can mount a host volume / docker volume and reuse downloads to speedup later runs.

So Cache will be enabled if BUILDPACK_CACHE points to an existing folder.

We should consider using AdoptOpenJDK for building all java version

api: https://api.adoptopenjdk.net/swagger-ui/

https://api.adoptopenjdk.net/v3/info/available_releases

{
  "available_lts_releases": [
    8,
    11
  ],
  "available_releases": [
    8,
    9,
    10,
    11,
    12,
    13,
    14
  ],
  "most_recent_feature_release": 14,
  "most_recent_lts": 11
}

https://api.adoptopenjdk.net/v3/info/release_versions?page=0&page_size=10&release_type=ga&sort_order=DESC&vendor=adoptopenjdk

    {
      "adopt_build_number": 1,
      "build": 9,
      "major": 13,
      "minor": 0,
      "openjdk_version": "13.0.1+9",
      "security": 1,
      "semver": "13.0.1+9.1"
    },
    {
      "adopt_build_number": 1,
      "build": 33,
      "major": 13,
      "minor": 0,
      "openjdk_version": "13+33",
      "security": 0,
      "semver": "13.0.0+33.1"
    },

renovatebot/docker-buildpack#33 (comment)

Not all users of this buildpack will necessarily use containerbase/buildpack as their base image. For some, they will want to:

• Substitute their own compatible (Ubuntu) base image
• Copy over necessary buildpack files
• Perform installs

The questions I have are:

• Should the buildpack be published independently, e.g. as a GitHub release archive, so that it can be used/copied without needing containerbase/buildpack? I think this wouldn't be necessary, and it would complicate our own process
• Can we someone reduce the necessary copy/bootstrap lines?

This is currently how such a scenario works:

FROM renovate/buildpack:4@sha256:bc40fb3708fb004ecd8b3a4f9a91ecb790d84e26da0a50c5a003ee3b494cd64e AS buildpack

FROM ubuntu:20.04 as base

# Set env and shell
ENV BASH_ENV=/usr/local/etc/env
SHELL ["/bin/bash" , "-c"]

# Set up buildpack
COPY --from=buildpack /usr/local/bin/ /usr/local/bin/
COPY --from=buildpack /usr/local/buildpack/ /usr/local/buildpack/
RUN install-buildpack

We should document this, and accept improvements to optimize it if people come up with them.

We should defer git updates for a few days to allow binaries to have been published after the git tag was pushed.

• renovatebot/renovate#12707
• renovatebot/renovate#13463

• create prepare-tool
• allow user install

https://github.com/containerbase/buildpack/blob/2d2090ee8a8c723b917a15d216114834e32286fd/src/usr/local/buildpack/tools/php.sh#L33-L54

needed for new scoop package manager

ref renovatebot/renovate#7876

Minimum support: Simple way of adding a custom certificate chain to a buildpack-based build. Ideally we can describe a single format of file that all languages/managers recognize. If not then a way we can "transform" it to a different format if some need it.

Ideal support: ability map a certificate into a container, configure it with an env variable, and it "just works".

Today we see paths like this:

/usr/local/helm/3.7.1
/usr/local/dotnet
/home/ubuntu/.gem-global/bin
/home/ubuntu/.gem/ruby/3.0.0/bin
/usr/local/ruby/3.0.3/bin
/home/ubuntu/.cargo/bin
/usr/local/rust/1.57.0/bin
/usr/local/go/1.17.4/bin
/go/bin
/usr/local/elixir/1.13.0/bin
/home/ubuntu/.npm-global/bin
/usr/local/yarn/1.22.17/bin
/home/ubuntu/.local/bin
/usr/local/python/3.10.0/bin
/usr/local/poetry/1.1.12/bin
/usr/local/pnpm/6.23.1/bin
/usr/local/php/7.4.26/bin
/usr/local/node/14.18.2/bin
/usr/local/java/11.0.13+8/bin
/usr/local/gradle/6.9.1/bin
/usr/local/composer/2.1.14/bin
/home/ubuntu/bin
/home/ubuntu/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin

Can we restructure how we install so that it can be like this?

/home/ubuntu/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin

By this I'm hoping what can happen is that every time install-tool installs a version of a too, it symlinks it to /usr/local/bin or /home/ubuntu/bin.

e.g. install-tool npm 8.0.0 might install it into /home/ubuntu/.npm-global/bin but then there's be a symlink from /home/ubuntu/.npm-global/bin/npm to /home/ubuntu/bin/npm.

If we don't have to think about paths and dynamic ENV injection then wouldn't it be much easier to know what's installed?

node (tricky, to not interferre with renovate required node)

• generic download helper
• generic extraction helper
• existance check
• multiple tool versions (python has support via python{major} and python{major}.{minor} executables`

We should check / add and fix any linting issues on all buildpack repos.

• use husky, lint-staged, prettier, markdownlint
• optional linter: shellcheck, eslint (depends on repo)
• run shellcheck as pull_request_target, so it can send build status

This is also an uber issue for all repos. 🙃

TODO:

• Ensure node version on lint workflows
• containerbase/ruby-prebuild#10
• containerbase/php-prebuild#21
• containerbase/python-prebuild#11
• containerbase/erlang-prebuild#16

We want all tools to be able to support HTTP_PROXY, including with a wrapper if necessary. We also want to support that a buildpack-based Dockerfile can be built behind a HTTP_PROXY.

For each language we support (Java, Node, Python, etc) as well as the package managers used by Renovate (npm, yarn, Poetry, Bundler, etc) we should ideally have them benchmarked in this repo as part of our actions, e.g. so we know what the approximate time it takes to add each language/tool.

This will be particularly useful when we start looking into the feasibility of run-time installs.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Repository problems

These problems occurred while renovating this repository. View logs.

• WARN: Use matchDepNames instead of matchPackageNames

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

• chore(deps): update dependency conventional-changelog-conventionalcommits to v8
• chore(deps): update dependency semantic-release to v24
• chore(deps): update linters (major) (eslint, eslint-plugin-jest)
• chore(deps): update vitest monorepo to v2 (major) (@vitest/coverage-v8, @vitest/ui, vitest)
• fix(deps): update dependency @sindresorhus/is to v7
• fix(deps): update dependency execa to v9
• 🔐 Create all pending approval PRs at once 🔐

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• test(deps): update dependency checkov to v3.2.193
• test(deps): update dependency renovate to v37.432.0
• build(deps): lock file maintenance

Pending Status Checks

These updates await pending status checks. To force their creation now, click the checkbox below.

• chore(deps): update dependency @types/node to v20.14.11
• chore(deps): update dependency vite to v5.3.4
• chore(deps): update linters to v7.16.1 (@typescript-eslint/eslint-plugin, @typescript-eslint/parser)
• fix(deps): update dependency semver to v7.6.3
• chore(deps): update dependency husky to v9.1.0
• chore(deps): update dependency type-fest to v4.22.0
• fix(deps): update dependency pino to v9.3.1

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• test(deps): update dependency java to v21.0.4+7.0.lts
• chore(deps): update dependency typescript to v5.5.3
• test(deps): update elixir docker tag to v1.17.2
• test(deps): update dependency erlang to v27
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• chore(deps): update ubuntu docker tag to v24
• test(deps): update dart docker tag to v3
• test(deps): update dependency node to v20

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

And deprecate install-tool + npm, pnpm, yarn, lerna?

I think our images have previously aimed at v3 OpenShift compatibility but may not be v4 compatible. In particular we set USER 1000 but may rely on hardcoded /home/x. For some of our tools we rely on them finding config in their home directory.

I found a discussion prior to v4 which gives some hope that we might still be able to put them in /home/x and it works despite random UIDs: openshift/origin#23369 (comment)

If as that comment indicates we can set HOME=/home/x and ensure it's writable by group 0 then it could be enough for tools to find their "home directory config" regardless of what UID they run as.

Another question remains though: is it forbidden to set USER 1000 in our images for OpenShift compatibility as seemed to be recommended in OpenShift v3, or we just need to be aware of its limitations.

Some users will build inside a corporate/restricted network and need to explicitly allow hosts. Without help, it's a slow process for them to keep restarting the build, waiting for it to file, then trying to work out which host was blocked before restarting.

Ideally we could combine #2 with logging so that we could log all hosts we connect to during the build.

Challenges:

• Do we log the DNS lookups by the build container, or by the proxy? I'm thinking the build container only needs to do a DNS lookup for the proxy and otherwise the proxy does DNS lookups
• We typically cache builds for efficiency, but in that case how do we know the "complete" list of DNS lookups?