googlecloudplatform / python-runtime Goto Github PK

gcr.io/google_appengine/python - Docker images for python

Home Page: https://cloud.google.com/python

License: Apache License 2.0

Shell 29.75% Python 70.09% Dockerfile 0.16%

python-runtime's Introduction

Google Cloud Platform - Python Runtime Docker Image

This repository contains the source for the gcr.io/google-appengine/python docker base image. This image can be used as the base image for running applications on Google App Engine Flexible, Google Kubernetes Engine, or any other Docker host.

This image is based on Ubuntu Xenial and contains packages required to build most of the popular Python libraries. For more information about this runtime, see the documentation.

App Engine

When using App Engine Flexible, you can use the runtime without worrying about docker by specifying runtime: python in your app.yaml:

runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app

runtime_config:
  # You can also specify 2 for Python 2.7
  python_version: 3

If you have an existing App Engine application using this runtime and want to customize it, you can use the Cloud SDK to create a custom runtime:

gcloud beta app gen-config --custom

You can then modify the Dockerfile and .dockerignore as needed for you application.

Kubernetes Engine & other Docker hosts.

For other docker hosts, you'll need to create a Dockerfile based on this image that copies your application code, installs dependencies, and declares an command or entrypoint. For example:

FROM gcr.io/google-appengine/python

# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
# Use -p python3 or -p python3.7 to select python version. Default is version 2.
RUN virtualenv /env

# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH

# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
ADD requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt

# Add the application source code.
ADD . /app

# Run a WSGI server to serve the application. gunicorn must be declared as
# a dependency in requirements.txt.
CMD gunicorn -b :$PORT main:app

Building the image

Google regularly builds and releases this image at gcr.io/google-appengine/python.

See RELEASING.md for more information.

Contributing changes

See CONTRIBUTING.md

Licensing

See LICENSE

python-runtime's People

Contributors

Stargazers

Watchers

Forkers

proppy icecrime msgendev sgjesse gar1t rgbkrk kleopatra999 henryvps thiago picorb navreetaft vkaya tmatsuo olliewild grimderp maxssage rt-jirapong sharifelgamal dlorenc nkubala aatishnn androidguydd nvgr igorstimoli pombredanne walkward bigblah vikashdalal guidiniz liyanhui1228 pioti pardhee vivicode1 sweetlearn cynocracy digital83ninja mahavir1008 watunda vincentpanqi wsbaik zucay peterkuria2000 mstumberger priya-gittest priyankaardu chr1st0p ed00m imjasonh boyd18 jleibow27 john2912 evankanderson donmccasland yuhonghong7035 3838438-org-tools projectoperations chronix87 okj9dddd xingao267 maniarasan-selvam davecheez google38438 diophung goodmonsters eoinmcc gypsybud winerkong jcalan8907 edgargomero bbbbbgit coloro79 alpha326 shruthirames udoyen muskanmahajan37 sarthak-kaps isabella232 dajiaji global-localhost global19 global19-atlassian-net ryanbalfanz mrfusion-jp digitalempire00 vardaan-raj qtassurance zachariahward harshb36 prasanthr recommend-games mirairo heyalexchoi msuozzo divyapatil4554 edwinvincent6211 jiaxinggong chusakd renovate-bot magar51 chriss-0x01

python-runtime's Issues

CARS

Audit dash vs underscore everywhere

Docker image names sometimes treat - and _ the same, and sometimes not. We should pick one and always use it everywhere. For example:

$ docker images gcr.io/google-appengine/debian8:latest
REPOSITORY                        TAG                 IMAGE ID            CREATED             SIZE
gcr.io/google-appengine/debian8   latest              bce67a0a288d        2 weeks ago         128.1 MB

$ docker images gcr.io/google_appengine/debian8:latest
REPOSITORY                        TAG                 IMAGE ID            CREATED             SIZE
gcr.io/google_appengine/debian8   latest              9e348dc9669e        3 months ago        128 MB

System tests get permission errors, then hang

ERROR: test_detect_text_gcs (vision.TestVisionClientText)
----------------------------------------------------------------------

Forbidden: 403 Google Cloud Vision API has not been used in project cloud-python-runtime-qa before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/vision.googleapis.com/overview?project=cloud-python-runtime-qa then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry. (POST https://vision.googleapis.com/v1/images:annotate)

test_create_metric (logging_.TestLogging) ... Exception google.auth.exceptions.RefreshError: RefreshError('invalid_scope: Empty or missing scope not allowed.', u'{\n  "error" : "invalid_scope",\n  "error_description" : "Empty or missing scope not allowed.",\n  "stack_trace" : "com.google.security.lso.protocol.oauth2.common.OAuth2ErrorException: invalid_scope: Empty or missing scope not allowed.\\n\\tat com.google.security.lso.protocol.oauth2.common.OAuth2ErrorException$Builder.build(OAuth2ErrorException.java:162)\\n\\tat com.google.security.lso.auth.oauth2.token.GetTokenConverter.convertErrors(GetTokenConverter.java:269)\\n\\tat com.google.security.lso.auth.oauth2.token.GetTokenConverter.convertResponse(GetTokenConverter.java:193)\\n\\tat com.google.security.lso.auth.oauth2.token.OAuth2TokenApiAction.lambda$process$0(OAuth2TokenApiAction.java:47)\\n\\tat com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:218)\\n\\tat com.google.common.util.concurrent.AbstractTransformFuture$AsyncTransformFuture.doTransform(AbstractTransformFuture.java:207)\\n\\tat com.google.common.util.concurrent.AbstractTransformFuture.run(AbstractTransformFuture.java:137)\\n\\tat com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:403)\\n\\tat com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:905)\\n\\tat com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:816)\\n\\tat com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:658)\\n\\tat com.google.net.rpc3.client.AbstractRpcFuture.success(AbstractRpcFuture.java:114)\\n\\tat com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher$1.runInContext(RpcStub.java:847)\\n\\tat com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:461)\\n\\tat com.google.tracing.CurrentContext.runInContext(CurrentContext.java:276)\\n\\tat com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:320)\\n\\tat com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:312)\\n\\tat com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:458)\\n\\tat com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.runCallback(RpcStub.java:885)\\n\\tat com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.rpcFinished(RpcStub.java:895)\\n\\tat com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.success(RpcStub.java:874)\\n\\tat com.google.net.rpc3.impl.client.RpcClientInternalContext.runCallbacks(RpcClientInternalContext.java:1244)\\n\\tat com.google.net.rpc3.impl.client.RpcClientInternalContext.finishRpcAndNotifyApp(RpcClientInternalContext.java:1141)\\n\\tat com.google.net.rpc3.util.RpcInProcessConnector$LocalRpcChannel.afterFinishingActiveRpc(RpcInProcessConnector.java:1198)\\n\\tat com.google.net.rpc3.util.RpcInProcessConnector$LocalRpcChannel.finishRpc(RpcInProcessConnector.java:1120)\\n\\tat com.google.net.rpc3.util.RpcInProcessConnector$LocalRpcChannel.waitAndFinishRpc(RpcInProcessConnector.java:1080)\\n\\tat com.google.net.rpc3.util.RpcInProcessConnector$LocalRpcChannel.handleResponse(RpcInProcessConnector.java:1043)\\n\\tat com.google.net.rpc3.util.RpcInProcessConnector$ServerInternalContext$2.run(RpcInProcessConnector.java:1721)\\n\\tat com.google.net.eventmanager.AbstractFutureTask$Sync.innerRun(AbstractFutureTask.java:260)\\n\\tat com.google.net.eventmanager.AbstractFutureTask.run(AbstractFutureTask.java:121)\\n\\tat com.google.net.eventmanager.EventManagerImpl.runTask(EventManagerImpl.java:594)\\n\\tat com.google.net.eventmanager.EventManagerImpl.internalRunWorkerLoop(EventManagerImpl.java:1012)\\n\\tat com.google.net.eventmanager.EventManagerImpl.runWorkerLoop(EventManagerImpl.java:892)\\n\\tat com.google.net.eventmanager.WorkerThreadInfo.runWorkerLoop(WorkerThreadInfo.java:161)\\n\\tat com.google.net.eventmanager.EventManagerImpl$WorkerThread.run(EventManagerImpl.java:1877)\\n\\tSuppressed: java.util.concurrent.ExecutionException: \\n"\n}') in 'grpc._cython.cygrpc.plugin_get_metadata' ignored

Invalid base image name "gcr.io/google_appengine/python"

This repo recommends using FROM gcr.io/google_appengine/python as the base Docker image when running a custom-images on Managed VMs, but it no longer works.

According to the documentation, custom images should use FROM gcr.io/google_appengine/python-compat as the base image.

FWIW: The Dockerfile in this repo provides environment variables and dependencies which python-compat does not. That base image does not appear to have a publicly-available Dockerfile in which developers can determine the differences. It'd be helpful to have all of this cleared up.

See this SO discussion for more info:
http://stackoverflow.com/questions/34169058/google-managed-vm-module-stuck-in-reboot-loop

./build.sh should gcloud docker docker pull images when necessary

Some images can't be pulled via 'docker pull', they must be pulled with 'gcloud docker -- pull' because of permission issues. The build.sh could do this automatically.

Get Unrecognized directory type when: gcloud beta app gen-config --custom

Trying to follow the readme to generate default configs:

$ gcloud beta app gen-config --custom
ERROR: (gcloud.beta.app.gen-config) Unrecognized directory type: [/Users/admin/test]

The directory is empty and writable

Enable LTO when compiling Python interpreters

For Python 3.6 and above.

See previous discussion in #96

UPLOAD_TO_STAGING doesn't work

Output from Jenkins running jenkins_build.sh:

2017-01-04_22_14: digest: sha256:7cc23e7e489c798a67147b69d076c1667b0f4281b6f8c174f8571faf63230721 size: 33663
DONE
--------------------------------------------------------------------------------

ID                                    CREATE_TIME                DURATION  SOURCE                                                                                                             IMAGES                                           STATUS
04473501-50b4-4d1a-ae71-2869c2b665f5  2017-01-04T22:14:43+00:00  39M53S    gs://vm-runtime-deployment-qa_cloudbuild/source/1483568064.66_gcr.io_google_appengine_python:2017-01-04_22_14.tgz  gcr.io/google_appengine/python:2017-01-04_22_14  SUCCESS
Untagged: gcr.io/google_appengine/python:staging
Error response from daemon: could not find image: no such id: gcr.io/google_appengine/python:2017-01-04_22_14
Build step 'Execute shell' marked build as failure
Sending e-mails to: [email protected] [email protected]
Finished: FAILURE

I can manually tag by pulling the image from GCR to local, tagging it locally, and pushing it back to GCR. Surely there is a better way.

gcloud docker -- pull gcr.io/google-appengine/python:2017-01-04_22_14
gcloud docker -- tag gcr.io/google-appengine/python:2017-01-04_22_14 gcr.io/google-appengine/python:staging
gcloud docker -- push gcr.io/google-appengine/python:staging

Add honcho package to library test

Honcho is used by our samples

Add python3.5 to $PATH

Otherwise I have to hardcode /opt/python3.5/bin in 6 different places, and counting.

Alpine based branch

Any plans or thoughts on providing an alpine based image to allow for smaller image sizes?

Size comparisons:

gcr.io/google-appengine/python: 808 MB
python:alpine: 89.4 MB

Always update pip, virtualenv, setuptools etc packages in base runtime image

However, last time I tried, there was a problem with pip 9.0.1, so make sure that's been fixed.

Update requirements.txt on a regular basis

@jonparrott has a script to update dependencies, I should start using it or something like it.

https://github.com/GoogleCloudPlatform/repository-gardener

Add CODEOWNERS to this repository

Latest build is missing `python3` and `pip3` symlinks

Hello!

The latest build (2018-03-20-170502) is missing the usual python3 and pip3 symlinks which were present in earlier versions (e.g. 2018-02-13-202400). It looks like this happened in the switch to the new build-from-source approach in #182... hopefully not hard to fix though.

To replicate:

docker run --rm -it gcr.io/google-appengine/python:2018-02-13-202400 python3 --version
# result: 'Python 3.4.2'

docker run --rm -it gcr.io/google-appengine/python:latest python3 --version
# result: 'docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"python3\": executable file not found in $PATH": unknown.'

Upgrade pip version

We are facing an issue when installing tfx package through pip3 from inside the container (version 9.0.3):

# pip3 install tfx
Collecting tfx
  Downloading https://files.pythonhosted.org/packages/27/ef/07522116d0680934ade5622322af06ed55d9820c3c23a7d97b62db1438db/tfx-0.15.0-py3-none-any.whl (504kB)
    100% |████████████████████████████████| 512kB 2.2MB/s 
Collecting tensorflow<3,>=1.15 (from tfx)
  Could not find a version that satisfies the requirement tensorflow<3,>=1.15 (from tfx) (from versions: 0.12.1, 1.0.0, 1.0.1, 1.1.0rc0, 1.1.0rc1, 1.1.0rc2, 1.1.0, 1.2.0rc0, 1.2.0rc1, 1.2.0rc2, 1.2.0, 1.2.1, 1.3.0rc0, 1.3.0rc1, 1.3.0rc2, 1.3.0, 1.4.0rc0, 1.4.0rc1, 1.4.0, 1.4.1, 1.5.0rc0, 1.5.0rc1, 1.5.0, 1.5.1, 1.6.0rc0, 1.6.0rc1, 1.6.0, 1.7.0rc0, 1.7.0rc1, 1.7.0, 1.7.1, 1.8.0rc0, 1.8.0rc1, 1.8.0, 1.9.0rc0, 1.9.0rc1, 1.9.0rc2, 1.9.0, 1.10.0rc0, 1.10.0rc1, 1.10.0, 1.10.1, 1.11.0rc0, 1.11.0rc1, 1.11.0rc2, 1.11.0, 1.12.0rc0, 1.12.0rc1, 1.12.0rc2, 1.12.0, 1.12.2, 1.12.3, 1.13.0rc0, 1.13.0rc1, 1.13.0rc2, 1.13.1, 1.13.2, 1.14.0rc0, 1.14.0rc1, 1.14.0, 2.0.0a0, 2.0.0b0, 2.0.0b1)
No matching distribution found for tensorflow<3,>=1.15 (from tfx)
You are using pip version 9.0.3, however version 19.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

The same installation succeeds after upgrading pip through pip3 install --upgrade pip.

Is there a plan to have the pip version upgraded?

There was #201 which downgraded pip from 18.0 to 9.0.3, referencing pypa/pip#5240. A related issue pypa/pip#5599 is still open but it says that the problems are not with pip itself but due to incorrect use of pip. Perhaps they could be fixed or worked around in the pip installation in the Python runtime image?

@dlorenc @liyanhui1228

Official docs say this should be 3.7.2 so why is it 3.6?

The official docs for the flex environment say the version is 3.7.2 but clearly this repository is 3.6.

Build interpreters with make -j and see if it's faster

Building the interpreters is slow. It'd be nice to use make -j<num_jobs> to build with more cores.
We'd have to make sure it: 1) still produces correct results, 2) is actually faster, 3) what's the right value of num_jobs.

Currently Container Builder uses a single core VM for building, but is rolling out support for larger machine sizes, so we'll have to request an 8 or 32 core machine in cloudbuild.yaml to actually get a benefit.

Removal of Wheezy and Jessie (except LTS) from mirrors]

Files python-runtime/python-interpreter-builder/Dockerfile.in and python-runtime/runtime-image/Dockerfile.in mention that Google App Engine base image is debian:

# The Google App Engine base image is debian (jessie) with ca-certificates
# installed.

On March 20th 2019 the removal of Wheezy and Jessie (except LTS) from mirrors was announced. Now, when building a Docker image FROM gcr.io/google_appengine/python:latest, the build fails with:

: Ign http://httpredir.debian.org jessie InRelease
: Get:1 http://httpredir.debian.org jessie-updates InRelease [6572 B]
: Get:2 http://security.debian.org jessie/updates InRelease [44.2 kB]
: Get:3 http://httpredir.debian.org jessie Release.gpg [2420 B]
: Get:4 http://httpredir.debian.org jessie Release [148 kB]
: Get:5 http://httpredir.debian.org jessie/main amd64 Packages [9098 kB]
: Get:6 http://security.debian.org jessie/updates/main amd64 Packages [866 kB]
: W: Failed to fetch http://httpredir.debian.org/debian/dists/jessie-updates/InRelease
: Unable to find expected entry 'main/binary-amd64/Packages' in Release file
: Wrong sources.list entry or malformed file
: E: Some index files failed to download. They have been ignored, or old ones used instead.

Some more information on https://unix.stackexchange.com/a/508728

`gcloud preview app` will go away

Please use:
https://cloud.google.com/sdk/gcloud/reference/beta/app/gen-config
and
gcloud beta app gen-config

in the README

Improve credential handling in system test

Once we can avoid copying around credentials files in build.sh, we should. Currently blocked on internal changes to cloudbuild.

uwsgi fails to build/install

Output:

  Running setup.py install for uwsgi: started
    Running setup.py install for uwsgi: finished with status 'error'
    Complete output from command /env/bin/python3.5 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-7gpf1_lc/uwsgi/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-lvrz5yos-record/install-record.txt --single-version-externally-managed --compile --install-headers /env/include/site/python3.5/uwsgi:
    /opt/python3.5/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'descriptions'
      warnings.warn(msg)
    running install
    using profile: buildconf/default.ini
    detected include path: ['/usr/lib/gcc/x86_64-linux-gnu/4.9/include', '/usr/local/include', '/usr/lib/gcc/x86_64-linux-gnu/4.9/include-fixed', '/usr/include/x86_64-linux-gnu', '/usr/include']
    Patching "bin_name" to properly install_scripts dir
    detected CPU cores: 1
    configured CFLAGS: -O2 -I. -Wall -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -DUWSGI_HAS_IFADDRS -DUWSGI_ZLIB -DUWSGI_LOCK_USE_MUTEX -DUWSGI_EVENT_USE_EPOLL -DUWSGI_EVENT_TIMER_USE_TIMERFD -DUWSGI_EVENT_FILEMONITOR_USE_INOTIFY -DUWSGI_VERSION="\"2.0.15\"" -DUWSGI_VERSION_BASE="2" -DUWSGI_VERSION_MAJOR="0" -DUWSGI_VERSION_MINOR="15" -DUWSGI_VERSION_REVISION="0" -DUWSGI_VERSION_CUSTOM="\"\"" -DUWSGI_YAML -DUWSGI_SSL -I/usr/include/libxml2 -DUWSGI_XML -DUWSGI_XML_LIBXML2 -DUWSGI_PLUGIN_DIR="\".\"" -DUWSGI_DECLARE_EMBEDDED_PLUGINS="UDEP(python);UDEP(gevent);UDEP(ping);UDEP(cache);UDEP(nagios);UDEP(rrdtool);UDEP(carbon);UDEP(rpc);UDEP(corerouter);UDEP(fastrouter);UDEP(http);UDEP(ugreen);UDEP(signal);UDEP(syslog);UDEP(rsyslog);UDEP(logsocket);UDEP(router_uwsgi);UDEP(router_redirect);UDEP(router_basicauth);UDEP(zergpool);UDEP(redislog);UDEP(mongodblog);UDEP(router_rewrite);UDEP(router_http);UDEP(logfile);UDEP(router_cache);UDEP(rawrouter);UDEP(router_static);UDEP(sslrouter);UDEP(spooler);UDEP(cheaper_busyness);UDEP(symcall);UDEP(transformation_tofile);UDEP(transformation_gzip);UDEP(transformation_chunked);UDEP(transformation_offload);UDEP(router_memcached);UDEP(router_redis);UDEP(router_hash);UDEP(router_expires);UDEP(router_metrics);UDEP(transformation_template);UDEP(stats_pusher_socket);" -DUWSGI_LOAD_EMBEDDED_PLUGINS="ULEP(python);ULEP(gevent);ULEP(ping);ULEP(cache);ULEP(nagios);ULEP(rrdtool);ULEP(carbon);ULEP(rpc);ULEP(corerouter);ULEP(fastrouter);ULEP(http);ULEP(ugreen);ULEP(signal);ULEP(syslog);ULEP(rsyslog);ULEP(logsocket);ULEP(router_uwsgi);ULEP(router_redirect);ULEP(router_basicauth);ULEP(zergpool);ULEP(redislog);ULEP(mongodblog);ULEP(router_rewrite);ULEP(router_http);ULEP(logfile);ULEP(router_cache);ULEP(rawrouter);ULEP(router_static);ULEP(sslrouter);ULEP(spooler);ULEP(cheaper_busyness);ULEP(symcall);ULEP(transformation_tofile);ULEP(transformation_gzip);ULEP(transformation_chunked);ULEP(transformation_offload);ULEP(router_memcached);ULEP(router_redis);ULEP(router_hash);ULEP(router_expires);ULEP(router_metrics);ULEP(transformation_template);ULEP(stats_pusher_socket);"
    *** uWSGI compiling server core ***
    core/utils.o is up to date
    core/protocol.o is up to date
    core/socket.o is up to date
    core/logging.o is up to date
    core/master.o is up to date
    core/master_utils.o is up to date
    core/emperor.o is up to date
    core/notify.o is up to date
    core/mule.o is up to date
    core/subscription.o is up to date
    core/stats.o is up to date
    core/sendfile.o is up to date
    core/async.o is up to date
    core/master_checks.o is up to date
    core/fifo.o is up to date
    core/offload.o is up to date
    core/io.o is up to date
    core/static.o is up to date
    core/websockets.o is up to date
    core/spooler.o is up to date
    core/snmp.o is up to date
    core/exceptions.o is up to date
    core/config.o is up to date
    core/setup_utils.o is up to date
    core/clock.o is up to date
    core/init.o is up to date
    core/buffer.o is up to date
    core/reader.o is up to date
    core/writer.o is up to date
    core/alarm.o is up to date
    core/cron.o is up to date
    core/hooks.o is up to date
    core/plugins.o is up to date
    core/lock.o is up to date
    core/cache.o is up to date
    core/daemons.o is up to date
    core/errors.o is up to date
    core/hash.o is up to date
    core/master_events.o is up to date
    core/chunked.o is up to date
    core/queue.o is up to date
    core/event.o is up to date
    core/signal.o is up to date
    core/strings.o is up to date
    core/progress.o is up to date
    core/timebomb.o is up to date
    core/ini.o is up to date
    core/fsmon.o is up to date
    core/mount.o is up to date
    core/metrics.o is up to date
    core/plugins_builder.o is up to date
    core/sharedarea.o is up to date
    core/rpc.o is up to date
    core/gateway.o is up to date
    core/loop.o is up to date
    core/cookie.o is up to date
    core/querystring.o is up to date
    core/rb_timers.o is up to date
    core/transformations.o is up to date
    core/uwsgi.o is up to date
    proto/base.o is up to date
    proto/uwsgi.o is up to date
    proto/http.o is up to date
    proto/fastcgi.o is up to date
    proto/scgi.o is up to date
    proto/puwsgi.o is up to date
    lib/linux_ns.o is up to date
    core/zlib.o is up to date
    core/yaml.o is up to date
    core/ssl.o is up to date
    core/legion.o is up to date
    core/xmlconf.o is up to date
    [gcc -pthread] core/dot_h.o
    [gcc -pthread] core/config_py.o
    *** uWSGI compiling embedded plugins ***
    plugins/python/python_plugin.o is up to date
    plugins/python/pyutils.o is up to date
    plugins/python/pyloader.o is up to date
    plugins/python/wsgi_handlers.o is up to date
    plugins/python/wsgi_headers.o is up to date
    plugins/python/wsgi_subhandler.o is up to date
    plugins/python/web3_subhandler.o is up to date
    plugins/python/pump_subhandler.o is up to date
    plugins/python/gil.o is up to date
    plugins/python/uwsgi_pymodule.o is up to date
    plugins/python/profiler.o is up to date
    plugins/python/symimporter.o is up to date
    plugins/python/tracebacker.o is up to date
    plugins/python/raw.o is up to date
    plugins/gevent/gevent.o is up to date
    plugins/gevent/hooks.o is up to date
    plugins/ping/ping_plugin.o is up to date
    plugins/cache/cache.o is up to date
    plugins/nagios/nagios.o is up to date
    plugins/rrdtool/rrdtool.o is up to date
    plugins/carbon/carbon.o is up to date
    plugins/rpc/rpc_plugin.o is up to date
    plugins/corerouter/cr_common.o is up to date
    plugins/corerouter/cr_map.o is up to date
    plugins/corerouter/corerouter.o is up to date
    plugins/fastrouter/fastrouter.o is up to date
    plugins/http/http.o is up to date
    plugins/http/keepalive.o is up to date
    plugins/http/https.o is up to date
    plugins/http/spdy3.o is up to date
    plugins/ugreen/ugreen.o is up to date
    plugins/signal/signal_plugin.o is up to date
    plugins/syslog/syslog_plugin.o is up to date
    plugins/rsyslog/rsyslog_plugin.o is up to date
    plugins/logsocket/logsocket_plugin.o is up to date
    plugins/router_uwsgi/router_uwsgi.o is up to date
    plugins/router_redirect/router_redirect.o is up to date
    plugins/router_basicauth/router_basicauth.o is up to date
    plugins/zergpool/zergpool.o is up to date
    plugins/redislog/redislog_plugin.o is up to date
    plugins/mongodblog/mongodblog_plugin.o is up to date
    plugins/router_rewrite/router_rewrite.o is up to date
    plugins/router_http/router_http.o is up to date
    plugins/logfile/logfile.o is up to date
    plugins/router_cache/router_cache.o is up to date
    plugins/rawrouter/rawrouter.o is up to date
    plugins/router_static/router_static.o is up to date
    plugins/sslrouter/sslrouter.o is up to date
    plugins/spooler/spooler_plugin.o is up to date
    plugins/cheaper_busyness/cheaper_busyness.o is up to date
    plugins/symcall/symcall_plugin.o is up to date
    plugins/transformation_tofile/tofile.o is up to date
    plugins/transformation_gzip/gzip.o is up to date
    plugins/transformation_chunked/chunked.o is up to date
    plugins/transformation_offload/offload.o is up to date
    plugins/router_memcached/router_memcached.o is up to date
    plugins/router_redis/router_redis.o is up to date
    plugins/router_hash/router_hash.o is up to date
    plugins/router_expires/expires.o is up to date
    plugins/router_metrics/plugin.o is up to date
    plugins/transformation_template/tt.o is up to date
    plugins/stats_pusher_socket/plugin.o is up to date
    *** uWSGI linking ***
    gcc -pthread -o /env/bin/uwsgi  core/utils.o core/protocol.o core/socket.o core/logging.o core/master.o core/master_utils.o core/emperor.o core/notify.o core/mule.o core/subscription.o core/stats.o core/sendfile.o core/async.o core/master_checks.o core/fifo.o core/offload.o core/io.o core/static.o core/websockets.o core/spooler.o core/snmp.o core/exceptions.o core/config.o core/setup_utils.o core/clock.o core/init.o core/buffer.o core/reader.o core/writer.o core/alarm.o core/cron.o core/hooks.o core/plugins.o core/lock.o core/cache.o core/daemons.o core/errors.o core/hash.o core/master_events.o core/chunked.o core/queue.o core/event.o core/signal.o core/strings.o core/progress.o core/timebomb.o core/ini.o core/fsmon.o core/mount.o core/metrics.o core/plugins_builder.o core/sharedarea.o core/rpc.o core/gateway.o core/loop.o core/cookie.o core/querystring.o core/rb_timers.o core/transformations.o core/uwsgi.o proto/base.o proto/uwsgi.o proto/http.o proto/fastcgi.o proto/scgi.o proto/puwsgi.o lib/linux_ns.o core/zlib.o core/yaml.o core/ssl.o core/legion.o core/xmlconf.o core/dot_h.o core/config_py.o plugins/python/python_plugin.o plugins/python/pyutils.o plugins/python/pyloader.o plugins/python/wsgi_handlers.o plugins/python/wsgi_headers.o plugins/python/wsgi_subhandler.o plugins/python/web3_subhandler.o plugins/python/pump_subhandler.o plugins/python/gil.o plugins/python/uwsgi_pymodule.o plugins/python/profiler.o plugins/python/symimporter.o plugins/python/tracebacker.o plugins/python/raw.o plugins/gevent/gevent.o plugins/gevent/hooks.o plugins/ping/ping_plugin.o plugins/cache/cache.o plugins/nagios/nagios.o plugins/rrdtool/rrdtool.o plugins/carbon/carbon.o plugins/rpc/rpc_plugin.o plugins/corerouter/cr_common.o plugins/corerouter/cr_map.o plugins/corerouter/corerouter.o plugins/fastrouter/fastrouter.o plugins/http/http.o plugins/http/keepalive.o plugins/http/https.o plugins/http/spdy3.o plugins/ugreen/ugreen.o plugins/signal/signal_plugin.o plugins/syslog/syslog_plugin.o plugins/rsyslog/rsyslog_plugin.o plugins/logsocket/logsocket_plugin.o plugins/router_uwsgi/router_uwsgi.o plugins/router_redirect/router_redirect.o plugins/router_basicauth/router_basicauth.o plugins/zergpool/zergpool.o plugins/redislog/redislog_plugin.o plugins/mongodblog/mongodblog_plugin.o plugins/router_rewrite/router_rewrite.o plugins/router_http/router_http.o plugins/logfile/logfile.o plugins/router_cache/router_cache.o plugins/rawrouter/rawrouter.o plugins/router_static/router_static.o plugins/sslrouter/sslrouter.o plugins/spooler/spooler_plugin.o plugins/cheaper_busyness/cheaper_busyness.o plugins/symcall/symcall_plugin.o plugins/transformation_tofile/tofile.o plugins/transformation_gzip/gzip.o plugins/transformation_chunked/chunked.o plugins/transformation_offload/offload.o plugins/router_memcached/router_memcached.o plugins/router_redis/router_redis.o plugins/router_hash/router_hash.o plugins/router_expires/expires.o plugins/router_metrics/plugin.o plugins/transformation_template/tt.o plugins/stats_pusher_socket/plugin.o -lpthread -lm -rdynamic -ldl -lz -lssl -lcrypto -lxml2 -lpthread -ldl -lutil -lm /opt/python3.5/lib/python3.5/config-3.5m/libpython3.5m.a -lutil -lcrypt
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37040): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37048): undefined reference to `.L2'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37050): undefined reference to `.L3'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37058): undefined reference to `.L4'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37060): undefined reference to `.L5'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37068): undefined reference to `.L6'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37070): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37078): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37080): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37088): undefined reference to `.L7'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37090): undefined reference to `.L8'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37098): undefined reference to `.L9'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370a0): undefined reference to `.L10'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370a8): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370b0): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370b8): undefined reference to `.L11'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370c0): undefined reference to `.L12'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370c8): undefined reference to `.L13'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370d0): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370d8): undefined reference to `.L14'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370e0): undefined reference to `.L15'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370e8): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370f0): undefined reference to `.L16'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x370f8): undefined reference to `.L17'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37100): undefined reference to `.L18'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37108): undefined reference to `.L19'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37110): undefined reference to `.L20'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37118): undefined reference to `.L21'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37120): undefined reference to `.L22'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37128): undefined reference to `.L23'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37130): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37138): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37140): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37148): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37150): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37158): more undefined references to `.L1' follow
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x371d0): undefined reference to `.L24'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x371d8): undefined reference to `.L25'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x371e0): undefined reference to `.L26'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x371e8): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x371f0): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x371f8): undefined reference to `.L27'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37200): undefined reference to `.L28'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37208): undefined reference to `.L29'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37210): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37218): undefined reference to `.L30'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37220): undefined reference to `.L31'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37228): undefined reference to `.L32'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37230): undefined reference to `.L33'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37238): undefined reference to `.L34'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37240): undefined reference to `.L35'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37248): undefined reference to `.L36'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37250): undefined reference to `.L37'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37258): undefined reference to `.L38'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37260): undefined reference to `.L39'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37268): undefined reference to `.L40'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37270): undefined reference to `.L41'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37278): undefined reference to `.L42'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37280): undefined reference to `.L43'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37288): undefined reference to `.L44'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37290): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37298): undefined reference to `.L45'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372a0): undefined reference to `.L46'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372a8): undefined reference to `.L47'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372b0): undefined reference to `.L48'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372b8): undefined reference to `.L49'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372c0): undefined reference to `.L50'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372c8): undefined reference to `.L51'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372d0): undefined reference to `.L52'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372d8): undefined reference to `.L53'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372e0): undefined reference to `.L54'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372e8): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372f0): undefined reference to `.L55'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x372f8): undefined reference to `.L56'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37300): undefined reference to `.L57'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37308): undefined reference to `.L58'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37310): undefined reference to `.L59'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37318): undefined reference to `.L60'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37320): undefined reference to `.L61'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37328): undefined reference to `.L62'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37330): undefined reference to `.L63'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37338): undefined reference to `.L64'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37340): undefined reference to `.L65'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37348): undefined reference to `.L66'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37350): undefined reference to `.L67'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37358): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37360): undefined reference to `.L68'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37368): undefined reference to `.L69'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37370): undefined reference to `.L70'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37378): undefined reference to `.L71'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37380): undefined reference to `.L72'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37388): undefined reference to `.L73'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37390): undefined reference to `.L74'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37398): undefined reference to `.L75'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373a0): undefined reference to `.L76'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373a8): undefined reference to `.L77'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373b0): undefined reference to `.L78'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373b8): undefined reference to `.L79'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373c0): undefined reference to `.L80'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373c8): undefined reference to `.L81'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373d0): undefined reference to `.L82'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373d8): undefined reference to `.L83'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373e0): undefined reference to `.L84'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373e8): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373f0): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x373f8): undefined reference to `.L85'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37400): undefined reference to `.L86'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37408): undefined reference to `.L87'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37410): undefined reference to `.L88'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37418): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37420): undefined reference to `.L89'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37428): undefined reference to `.L90'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37430): undefined reference to `.L91'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37438): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37440): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37448): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37450): undefined reference to `.L92'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37458): undefined reference to `.L93'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37460): undefined reference to `.L94'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37468): undefined reference to `.L95'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37470): undefined reference to `.L96'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37478): undefined reference to `.L97'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37480): undefined reference to `.L98'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37488): undefined reference to `.L99'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37490): undefined reference to `.L100'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37498): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374a0): undefined reference to `.L101'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374a8): undefined reference to `.L102'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374b0): undefined reference to `.L103'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374b8): undefined reference to `.L104'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374c0): undefined reference to `.L105'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374c8): undefined reference to `.L106'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374d0): undefined reference to `.L107'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374d8): undefined reference to `.L108'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374e0): undefined reference to `.L109'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374e8): undefined reference to `.L110'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374f0): undefined reference to `.L111'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x374f8): undefined reference to `.L112'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37500): undefined reference to `.L113'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37508): undefined reference to `.L114'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37510): undefined reference to `.L115'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37518): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37520): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37528): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37530): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37538): undefined reference to `.L1'
    /tmp/ccFAvNLC.ltrans0.ltrans.o:(.data+0x37540): more undefined references to `.L1' follow
    collect2: error: ld returned 1 exit status
    *** error linking uWSGI ***

Benchmark the same interpreter from release to release

The current benchmarking code in tests/benchmark has code to benchmark Python 2.7 against 3.4, and 3.4 against 3.5, in the same Docker image.

We would like to also be able to benchmark two different Docker images against each other. For example, we'd like to be able to compare the Python 3.5.2 interpreter in gcr.io/google-appengine/python:2017-03-14_04_48 against the same interpreter in gcr.io/google-appengine/python:2017-04-07-173010

Dockerfile FROM caching

This statement

FROM gcr.io/google_appengine/debian8

Isn't guaranteed to pull in the latest image from GCR. I believe it just uses the latest image that happens to be on the local Docker daemon.

Build process not concurrency safe

If a single machine (say Jenkins) runs two 'make build|tests|benchmarks' concurrently (in two branches and/or directories), they will both try to use the Docker image tag 'google/python' and one will lose.

Question: Unable to install psycopg2

Hi again,

I had another question. I was trying to get Django up and running on the python-runtime image and I ran into an issue getting it working. I was planning on using postgres as my database, which means I need to install the psycopg2 python package, but the psycopg2 package relies on libpq-dev which I need to install using apt-get. So I thought I could just add a RUN statement to my Dockerfile and install the libpq-dev package, but that doesn't work because of the ONBUILD statements on the python-runtime image run before any of my statements run.

Dockerfile:

FROM google/python-runtime

RUN apt-get install libpq-dev

This will error during the pip install requirements.txt line.

Is this possible to do with the python-runtime image or should I just drop down into the google/python image and copy what I need from the python-runtime image?

Thanks,

gcloud utility not working

I failed to run gcloud utility (https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-120.0.0-linux-x86_64.tar.gz) with this image.

It seems like valuable requests get no response.
For instance, gcloud app versions list --log-http --verbosity debug

freezes/fails on services API:

--request-start--
uri: https://appengine.googleapis.com/v1beta5/apps/<project_id>/services?alt=json
method: GET
-headers-start-
Authorization: Bearer <token>
accept: application/json
accept-encoding: gzip, deflate
content-length: 0
user-agent: blablabla gcloud/120.0.0 command/gcloud.app.versions.list invocation-id/blablabla environment/GCE environment-version/None interactive/True python/2.7.9 (Linux 3.16.0-4-amd64)
-headers-end-
-body-start-
-body-end-
--request-end--
DEBUG: Caught HTTP error ResponseNotReady, retrying: 
DEBUG: Retrying request to url https://appengine.googleapis.com/v1beta5/apps/<project_id>/services?alt=json
 after exception 
--request-start--

The following command fails with similar symptoms
gcloud app services list

Any idea how to fix it?

Update benchmarking code

Running the benchmarks gives this:

+++++++++++
Deprecated!
+++++++++++

The old benchmarks project is now deprecated, please move
to the new shiny Python benchmark suite:

https://github.com/python/performance
https://pypi.python.org/pypi/performance

Record and display the results of benchmarking

The current benchmarking framework does not record the results anywhere. We would like to store the results of various benchmarks somewhere (for example, BigTable), and then visualize that information via a dashboard or similar.

Run "make test" when compiling interpreter

I thought we did this already, but we don't, except indirectly as part of the profile-guided optimization.

Update Python 3.5.2 to 3.5.3 in image

Switch from using local docker to Container Builder

This should be as simple as switching "docker build" in the Makefile to:

gcloud alpha container builds create . -t $IMAGE_NAME.

Update README.md and add CONTRIBUTING

README.md should contain license and contributing information.

python-interpreter-builder/Dockerfile FROM debian8 not base

This line in python-interpreter-builder/Dockerfile

FROM gcr.io/google_appengine/base

Should be

FROM gcr.io/google_appengine/debian8

Remove Cloud SDK install from system test

It might not be necessary, or it might be necessary just for one-time test setup, or it might be necessary now but unnecessary once we switch to some future CI system or Container Builder.

If it is still necessary, there might be a better way to download it.

Add Python 3.6.1 interpreter to image

Include python 3.8 interpreter.

Any chance we can get the shiny, new Python version?

Figure out Python 3.6 and openssl deprecation

The Python 3.6 documentation says building with OpenSSL 1.0.1 is deprecated. Figure out what this means, and what we need to do about it.

https://docs.python.org/3/library/ssl.html

RPCFailedError: The remote RPC to the application server failed for call taskqueue.BulkAdd()

I am sporadically getting the following error when using the deferred library to add a task to the taskqueue. I am running on the App Engine Flexible / Managed VM environment using the gcr.io/google_appengine/python-compat-multicore Docker image. This SO post seems to describe a similar issue, but does not have much more information.

Any thoughts?

Traceback (most recent call last):
  ...
  File "/home/vmagent/app/libs/../updater/CrossPlatform.py", line 589, in Defer
    defer.defer(fn_to_run, *args, **kwargs)
  File "/home/vmagent/app/defer/api.py", line 12, in defer
    default_backend(func, args, kwargs, **config)
  File "/home/vmagent/app/defer/backends/appengine.py", line 12, in defer
    gae_defer(func, *args, **kwargs)
  File "/env/local/lib/python2.7/site-packages/google/appengine/ext/deferred/deferred.py", line 269, in defer
    return task.add(queue, transactional=transactional)
  File "/env/local/lib/python2.7/site-packages/google/appengine/api/taskqueue/taskqueue.py", line 1134, in add
    return self.add_async(queue_name, transactional).get_result()
  File "/env/local/lib/python2.7/site-packages/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
    return self.__get_result_hook(self)
  File "/env/local/lib/python2.7/site-packages/google/appengine/api/taskqueue/taskqueue.py", line 1948, in ResultHook
    rpc.check_success()
  File "/env/local/lib/python2.7/site-packages/google/appengine/api/apiproxy_stub_map.py", line 579, in check_success
    self.__rpc.CheckSuccess()
  File "/env/local/lib/python2.7/site-packages/google/appengine/ext/vmruntime/vmstub.py", line 312, in _WaitImpl
    raise self._ErrorException(*_DEFAULT_EXCEPTION)
RPCFailedError: The remote RPC to the application server failed for call taskqueue.BulkAdd().

Run google-cloud-python system tests under Python 3.5

In system_tests/Dockerfile, this line:

tox -e system-tests,system-tests3 -- ${MODULES}

Ideally by adding a tox target as tracked here: googleapis/google-cloud-python#2670

Or we could do the virtualenv/path setup ourselves.

GAP Flexible environment - Python

Hi, we are using GAP with flexible environments in production but now we cannot do a new deployment of our projects/products. Using the latest version of this container all our deployments are failed.
We are getting an error when container tries to install the dependencies but we don't have any new dependency, in our code base, and our last successful deployments were hours before of this last container released from your part. Any idea / suggest ?. Here the traceback that we are getting:

ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: exit status 2
ERROR
The command '/bin/sh -c pip install -r /app/deploy-requirements.txt' returned a non-zero code: 2
ValueError: too many values to unpack (expected 3)
for fpath, digest, length in old_csv_rows:
File "/env/lib/python3.6/site-packages/pip/_internal/wheel.py", line 565, in get_csv_rows_for_installed
outrows = get_csv_rows_for_installed(reader)
File "/env/lib/python3.6/site-packages/pip/_internal/wheel.py", line 583, in move_wheel_files
warn_script_location=warn_script_location,
File "/env/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 445, in move_wheel_files
use_user_site=use_user_site, pycompile=pycompile,
File "/env/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 919, in install
**kwargs
File "/env/lib/python3.6/site-packages/pip/_internal/req/__init__.py", line 57, in install_given_reqs
use_user_site=options.use_user_site,
File "/env/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 393, in run
status = self.run(options, args)
File "/env/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 176, in main
Traceback (most recent call last):
Exception:

Thanks.

Don't hardcode project name for system tests

Currently hardcoded to cloud-python-runtime-qa in the Dockerfile.

ca-certificates is either missing or outdated, causing pip install to fail when using git urls.

If you use a git+https url in your requirements.txt, for example:

-e git+https://github.com/GoogleCloudPlatform/gcloud-python.git#egg=gcloud

Then building the image will fail because pip can't verify the cert from github. It appears the root of this problem is ca-certificates is missing or invalid. Updating these fixes the problem.

Question: Why is there a virtualenv?

Sorry for opening an issue just to ask a question, but I was just curious why you install virtualenv at all inside of a container. I am just learning about Docker so it's completely possible that I just missed something here, so please forgive me.

I was under the impression that the purpose of virtualenv was to isolate packages installed for a specific application from the global python installation, thus avoiding dependency conflicts and in general keeping the system clean. However, each Docker image is it's own system. There is only one application in the system, what does it matter if that application installs packages in the global python installation? Isn't virtualenv superfluous in this case?

The only reason that I could think of is if you put the virtualenv on it's own volume and that way you could persist it (thus making successive deployments quicker). But I'm not even sure that's possible to do in a later image if the /env directory was already set up in the python-runtime image.

Thank you for your time.

Problem installing GDAL

Hello, I have some problem to configure GDAL in my Python environment.

In the "requirement.txt" I specified "GDAL==2.4.2" and
as pre-condition I configured Docker in order to install the needed system dependencies:
RUN apt-get update && apt-get install -y \ binutils \ gdal-bin \ python-gdal

During the installation process I get the following error:
Building wheel for GDAL (setup.py): started Building wheel for GDAL (setup.py): finished with status 'error' ERROR: Command errored out with exit status 1: command: /env/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-48kodxl1/GDAL/setup.py'"'"'; __file__='"'"'/tmp/pip-install-48kodxl1/GDAL/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-okgsrlrf --python-tag cp37 cwd: /tmp/pip-install-48kodxl1/GDAL/ Complete output (33 lines): WARNING: numpy not available! Array support will not be enabled running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-3.7 copying gdal.py -> build/lib.linux-x86_64-3.7
.
.
compilation terminated. error: command "x86_64-linux-gnu-gcc" failed with exit status 1 ERROR: Failed building wheel for GDAL

Is something missing in the base configuration?

thanks in advance.

Outdated `setuptools` causing package installation issues

Installing python-memcached==1.58 inside app engine started breaking for me today.

Running gcloud app deploy gives decent output up until the installation of python-memcached

The following output is what I get at that point

This fails on the docker build step RUN pip install -r requirements.txt

Upgrading setuptools to setuptools-36.5.0 fixed this problem. If I replicate the Docker build steps that run for app engine but add a line RUN pip install -U pip setuptools before running the pip install above, this is resolved completely.

This is the related issue in the python-memcached repo - linsomniac/python-memcached#103

How would we tackle this? Is it an option to upgrade the setuptools version? Do we pin it to a version? Or do you think it's acceptable keeping a broad version unpinned upgrade?

# Checkout the latest release.
RUN git checkout $(git describe --tags --abbrev=0)