python: base dependencies about base HOT 16 CLOSED

Great. I'll take a look at it.

from base.

@Chumper I think we can now add those dependencies to finish python

from base.

@wastrachan Please add your findings here 🙃

from base.

Background

I'm going to base my findings off of my reproduction repository: https://github.com/wastrachan/renovate-reproduction.
This repository contains a pipfile with a single dependency, mysqlclient.

mysqlclient relies on local mysql development headers to build (libmysqlclient-dev or libmariadb-dev). As these headers are missing from the base buildpack image for renovate -- any attempts by renovate to re-lock a Pipefile error out: https://github.com/wastrachan/renovate-reproduction/pull/2

Assumptions

It's been presented there is some desire to keep some parity with between the base renovate image and the official python images (as far as dependencies go). The -slim python images contain very few build dependencies, and are generally not held out as having any dependencies other than those required to build python:

This image does not contain the common packages contained in the default tag and only contains the minimal packages needed to run python. Unless you are working in an environment where only the python image will be deployed and you have space constraints, we highly recommend using the default image of this repository.

The base image of python then, is what we'd be looking at. The base image for python inherits buildpack-deps:

This tag is based off of buildpack-deps. buildpack-deps is designed for the average user of Docker who has many images on their system. It, by design, has a large number of extremely common Debian packages. This reduces the number of packages that images that derive from it need to install, thus reducing the overall size of all images on your system.

buildpack-deps, in turn, contains the required MySQL development headers (https://github.com/docker-library/buildpack-deps/blob/master/ubuntu/focal/Dockerfile#L56)

Conclusions

The simplest solution (particularly if there is a desire to maintain parity with the official python docker images) would be to add libmysqlclient-dev to the base image.

build-essential is not explicitly required, but a gcc compiler must be available to build mysqlclient.

So either: libmysqlclient-dev and gcc (159MB) would need to be added, OR libmysql-client-dev and build-essential (211MB).

This is a theoretical solution: #99

from base.

this is now unblocked

from base.

If it's enough, we should install default-libmysqlclient-dev (instead of libmysqlclient-dev) and gcc

from base.

@wastrachan Can you please recreate the reproduction, so we can fork it? I like to have a test repo. Maybe you can also past required files as gist here.

from base.

@viceice please install both, mysql and postgresql client dependencies, as they are the most common databases for python development.

from base.

Is there anything I can help with to get this over the finish line?

from base.

PRs are welcome. @viceice would this be done as part of the prepare step?

from base.

yes

from base.

@viceice PR opened in #594

from base.

It would be appreciated if this can be resolved.

I can help testing. We're using the PostgreSQL drivers.

from base.

@hugoboos you can open a new PR based on the existing and do the required changes from comments and discussions.

from base.

@viceice Thank you!

from base.

@viceice Thank you! ❤️

from base.