Giter Club home page Giter Club logo

docker-thumbor's Introduction

Minimal Usage

$ docker run -p 8000:8000 apsl/thumbor
$ wget http://localhost:8000/unsafe/300x300/i.imgur.com/bvjzPct.jpg

Docker thumbor and remotecv

Docker Pulls Docker Stars Build Status

Docker image for thumbor, and separated one for remotecv, necessary for the lazy detection.
All parameters of the thumbor config can be set with env vars.
The thumbor's docker expose port 8000 with the service.
Consider to use the docker-thumbor-nginx image to use nginx like a first cache.
The nginx cache check if the file exists in a shared volume (file_storage) after to send the request to thumbor (automated failover).
We propose two thumbor images aspl/thumbor and apsl/thumbor-multiprocess.
The first one (monoprocess) to use under a docker organization tool and the second one use circus to increase the number of thumbor processes. Use multiprocess if you need to deploy in one host and scale up.
The remotecv could be scaled increasing the number of docker images using the same redis queue.

Description

Docker thumbor image

  • All configuration via environment variables
  • Opencv detectors
  • Optimizer JPEG included
  • Nginx cache to optimize the server of result images in a separated docker image
  • Separated remotecv docker image, with AWS S3 loader support.
  • Use the official python images and best practices
  • A lot of docker-compose examples with tipical uses cases
  • envtpl to setup config files on start time, based on environ vars. https://github.com/andreasjansson/envtpl

Ports

  • 8000: thumbor

Docker-compose examples

Check the docker-compose examples:

Env vars and default value:

THUMBOR_LOG_FORMAT='%(asctime)s %(name)s:%(levelname)s %(message)s'
THUMBOR_LOG_DATE_FORMAT='%Y-%m-%d %H:%M:%S'
MAX_WIDTH=0
MAX_HEIGHT=0
MIN_WIDTH=1
MIN_HEIGHT=1
ALLOWED_SOURCES=[]
QUALITY=80
WEBP_QUALITY=None
PNG_COMPRESSION_LEVEL=6
AUTO_WEBP=False
MAX_AGE=86400
MAX_AGE_TEMP_IMAGE=0
RESPECT_ORIENTATION=False
IGNORE_SMART_ERRORS=False
PRESERVE_EXIF_INFO=False
ALLOW_ANIMATED_GIFS=True
USE_GIFSICLE_ENGINE=False
USE_BLACKLIST=False
LOADER='thumbor.loaders.http_loader'
STORAGE='thumbor.storages.file_storage'
AWS_ACCESS_KEY_ID='' -> Note: New in version 5.2.1c
AWS_SECRET_ACCESS_KEY='' -> Note: New in version 5.2.1c
RESULT_STORAGE='thumbor.result_storages.file_storage'
ENGINE='thumbor.engines.pil'
SECURITY_KEY='MY_SECURE_KEY'
ALLOW_UNSAFE_URL=True
ALLOW_OLD_URLS=True
FILE_LOADER_ROOT_PATH='/data/loader'
HTTP_LOADER_CONNECT_TIMEOUT=5
HTTP_LOADER_REQUEST_TIMEOUT=20
HTTP_LOADER_FOLLOW_REDIRECTS=True
HTTP_LOADER_MAX_REDIRECTS=5
HTTP_LOADER_FORWARD_USER_AGENT=False
HTTP_LOADER_DEFAULT_USER_AGENT='Thumbor/5.2.1'
HTTP_LOADER_PROXY_HOST=None
HTTP_LOADER_PROXY_PORT=None
HTTP_LOADER_PROXY_USERNAME=None
HTTP_LOADER_PROXY_PASSWORD=None
HTTP_LOADER_CA_CERTS=None
HTTP_LOADER_VALIDATE_CERTS=True
HTTP_LOADER_CLIENT_KEY=None
HTTP_LOADER_CLIENT_CERT=None
HTTP_LOADER_CURL_ASYNC_HTTP_CLIENT=False
STORAGE_EXPIRATION_SECONDS=2592000
STORES_CRYPTO_KEY_FOR_EACH_IMAGE=False
FILE_STORAGE_ROOT_PATH='/data/storage'
UPLOAD_MAX_SIZE=0
UPLOAD_ENABLED=False
UPLOAD_PHOTO_STORAGE='thumbor.storages.file_storage'
UPLOAD_DELETE_ALLOWED=False
UPLOAD_PUT_ALLOWED=False
UPLOAD_DEFAULT_FILENAME='image'
MONGO_STORAGE_SERVER_HOST='mongo'
MONGO_STORAGE_SERVER_PORT='27017'
MONGO_STORAGE_SERVER_DB='thumbor'
MONGO_STORAGE_SERVER_COLLECTION='images'
REDIS_STORAGE_SERVER_HOST='redis'
REDIS_STORAGE_SERVER_PORT='6379'
REDIS_STORAGE_SERVER_DB=0
REDIS_STORAGE_SERVER_PASSWORD=None
REDIS_RESULT_STORAGE_SERVER_HOST='redis'
REDIS_RESULT_STORAGE_SERVER_PORT=6379
REDIS_RESULT_STORAGE_SERVER_DB=0
REDIS_RESULT_STORAGE_SERVER_PASSWORD=None
MEMCACHE_STORAGE_SERVERS=['localhost:11211',]
MIXED_STORAGE_FILE_STORAGE='thumbor.storages.no_storage'
MIXED_STORAGE_CRYPTO_STORAGE='thumbor.storages.no_storage'
MIXED_STORAGE_DETECTOR_STORAGE='thumbor.storages.no_storage'
META_CALLBACK_NAME=None
DETECTORS=[]
FACE_DETECTOR_CASCADE_FILE='haarcascade_frontalface_alt.xml'
OPTIMIZERS=[]
JPEGTRAN_PATH='/usr/bin/jpegtran'
PROGRESSIVE_JPEG=True
FILTERS=['thumbor.filters.brightness', 'thumbor.filters.contrast', 'thumbor.filters.rgb', 'thumbor.filters.round_corner', 'thumbor.filters.quality', 'thumbor.filters.noise', 'thumbor.filters.watermark', 'thumbor.filters.equalize', 'thumbor.filters.fill', 'thumbor.filters.sharpen', 'thumbor.filters.strip_icc', 'thumbor.filters.frame', 'thumbor.filters.grayscale', 'thumbor.filters.rotate', 'thumbor.filters.format', 'thumbor.filters.max_bytes', 'thumbor.filters.convolution', 'thumbor.filters.blur', 'thumbor.filters.extract_focal', 'thumbor.filters.no_upscale']
RESULT_STORAGE_EXPIRATION_SECONDS=0
RESULT_STORAGE_FILE_STORAGE_ROOT_PATH='/data/result_storage'
RESULT_STORAGE_STORES_UNSAFE=False
REDIS_QUEUE_SERVER_HOST='redis'
REDIS_QUEUE_SERVER_PORT='6379'
REDIS_QUEUE_SERVER_DB='0'
REDIS_QUEUE_SERVER_PASSWORD=None
SQS_QUEUE_KEY_ID=None
SQS_QUEUE_KEY_SECRET=None
SQS_QUEUE_REGION='us-east-1'
USE_CUSTOM_ERROR_HANDLING=False
ERROR_HANDLER_MODULE='thumbor.error_handlers.sentry'
ERROR_FILE_LOGGER=None
ERROR_FILE_NAME_USE_CONTEXT='False'
SENTRY_DSN_URL=''
TC_AWS_REGION='eu-west-1'
TC_AWS_ENDPOINT=None
TC_AWS_STORAGE_BUCKET=''
TC_AWS_STORAGE_ROOT_PATH=''
TC_AWS_LOADER_BUCKET=''
TC_AWS_LOADER_ROOT_PATH=''
TC_AWS_RESULT_STORAGE_BUCKET=''
TC_AWS_RESULT_STORAGE_ROOT_PATH=''
TC_AWS_STORAGE_SSE=False
TC_AWS_STORAGE_RRS=False
TC_AWS_ENABLE_HTTP_LOADER=False
TC_AWS_ALLOWED_BUCKETS=False
TC_AWS_STORE_METADATA=False

docker-thumbor's People

Contributors

beenanner avatar braisgabin avatar cerber717 avatar eduherraiz avatar emazzotta avatar kkopachev avatar mcfedr avatar mynameiswhm avatar paurullan avatar psyduck787 avatar shaneog avatar spantaleev avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-thumbor's Issues

The docker container has insufficient SSL certificates

As of 6.0.1, the SSL certificates available to thumbor inside the container are insufficient to fetch some of the ssl based images (urls below). Bashing into the container and curl complains about not found certificates.

Curl:

root@deb08d83e7dc:~# curl  https://static01.nyt.com/images/2016/05/24/world/24FINLAND-web1/24FINLAND0-web1-superJumbo.jpg > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

Thumbor( http://192.168.99.100:8000/unsafe/300x200/smart/https%3A%2F%2Fstatic01.nyt.com%2Fimages%2F2016%2F05%2F24%2Fworld%2F24FINLAND-web1%2F24FINLAND0-web1-superJumbo.jpg):

2016-05-31 12:07:14 tornado.general:WARNING SSL Error on 8 ('23.37.97.37', 443): [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
2016-05-31 12:07:14 thumbor:WARNING ERROR retrieving image https://static01.nyt.com/images/2016/05/24/world/24FINLAND-web1/24FINLAND0-web1-superJumbo.jpg : [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
2016-05-31 12:07:14 tornado.access:ERROR 504 GET /unsafe/300x200/smart/https%3A%2F%2Fstatic01.nyt.com%2Fimages%2F2016%2F05%2F24%2Fworld%2F24FINLAND-web1%2F24FINLAND0-web1-superJumbo.jpg%20 (192.168.99.1) 58.22ms

If this cert bundle from here is used with --cacert it is able to fetch. (Not sure if adding that explicitly to the docker build file would be good idea.)

Image urls:

Thumbor S3 not working with Frankfurt Servers requiring API v4

I successfully had an image generated by thumbor on our amazon machine but it doesn't get uploaded to S3 due to this error:

thumbor_1 | 2016-03-16 10:05:22 thumbor:WARNING [STORAGE] error occured while storing data: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

This is likely due to our infrastructure being in Frankfurt which was rolled out after 2014 and thus requires a new authentication mechanism.

The S3 key and secret I generated work fine.

Could this please be implemented?

No module named redis_storage

We get this error when we tried to use thumbor 6.x (tried in 6.1.2, 6.1.1 and 6.0.1).
We use redis for remotecv.

This is out compose(docker cloud config)

loadbalancer:
  image: 'dockercloud/haproxy:1.5.3'
  deployment_strategy: high_availability
  environment:
    - BALANCE=uri
    - MODE=tcp
  expose:
    - '443'
  links:
    - nginx
  ports:
    - '80:80'
    - '1936:1936'
  restart: always
  roles:
    - global
  tags:
    - thumbor-prod
  target_num_containers: 2
nginx:
  image: 'apsl/thumbor-nginx:6.1.1'
  deployment_strategy: every_node
  links:
    - thumbor
  tags:
    - thumbor-prod
  volumes_from:
    - thumbor
thumbor:
  image: 'apsl/thumbor:6.1.1'
  deployment_strategy: every_node
  environment:
    - ALLOW_UNSAFE_URL=False
    - 'DETECTORS=[''thumbor.detectors.queued_detector.queued_complete_detector'']'
    - MAX_AGE=31556926
    - MAX_AGE_TEMP_IMAGE=84600
    - MIXED_STORAGE_DETECTOR_STORAGE=thumbor.storages.redis_storage
    - 'OPTIMIZERS=[''thumbor.optimizers.jpegtran'']'
    - REDIS_QUEUE_SERVER_DB=0
    - REDIS_QUEUE_SERVER_HOST=thumbor-redis
    - REDIS_QUEUE_SERVER_PORT=6379
    - REDIS_STORAGE_SERVER_DB=0
    - REDIS_STORAGE_SERVER_HOST=thumbor-redis
    - REDIS_STORAGE_SERVER_PORT=6379
    - RESULT_STORAGE=thumbor.result_storages.file_storage
    - RESULT_STORAGE_STORES_UNSAFE=False
    - SECURITY_KEY=something
    - STORAGE=thumbor.storages.mixed_storage
    - STORAGE_EXPIRATION_SECONDS=864000
  links:
    - thumbor-redis
  restart: always
  tags:
    - thumbor-prod
  volumes:
    - '/thumbor_store:/data/result_storage'
    - '/logs:/logs'
thumbor-redis:
  image: 'redis:latest'
  deployment_strategy: high_availability
  ports:
    - '6379'
  restart: always
  tags:
    - thumbor-prod
thumbor-remotecv:
  image: 'apsl/remotecv:6.1.1'
  deployment_strategy: every_node
  environment:
    - REDIS_DATABASE=0
    - REDIS_HOST=thumbor-redis
    - REDIS_PORT=6379
  links:
    - thumbor-redis
  restart: always
  tags:
    - thumbor-remotecv-prod

And this is the full error:

2016-09-05 13:57:52 tornado.application:ERROR Future exception was never retrieved: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1024, in run
    yielded = self.gen.send(value)
  File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 211, in get_image
    self.filters_runner.apply_filters(thumbor.filters.PHASE_AFTER_LOAD, transform)
  File "/usr/local/lib/python2.7/site-packages/thumbor/filters/__init__.py", line 81, in apply_filters
    callback()
  File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 209, in transform
    self.context.transformer.transform(after_transform_cb)
  File "/usr/local/lib/python2.7/site-packages/thumbor/transformer.py", line 102, in transform
    self.smart_detect()
  File "/usr/local/lib/python2.7/site-packages/thumbor/transformer.py", line 151, in smart_detect
    self.do_smart_detection().result()
  File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 237, in result
    raise_exc_info(self._exc_info)
  File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1021, in run
    yielded = self.gen.throw(*exc_info)
  File "/usr/local/lib/python2.7/site-packages/thumbor/transformer.py", line 174, in do_smart_detection
    focal_points = yield gen.maybe_future(self.context.modules.storage.get_detector_data(self.smart_storage_key))
  File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1015, in run
    value = future.result()
  File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 237, in result
    raise_exc_info(self._exc_info)
  File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 285, in wrapper
    yielded = next(result)
  File "/usr/local/lib/python2.7/site-packages/thumbor/storages/mixed_storage.py", line 73, in get_detector_data
    self._init_detector_storage()
  File "/usr/local/lib/python2.7/site-packages/thumbor/storages/mixed_storage.py", line 48, in _init_detector_storage
    class_name='Storage'
  File "/usr/local/lib/python2.7/site-packages/thumbor/importer.py", line 95, in import_item
    module = self.import_class('%s.%s' % (conf_value, class_name))
  File "/usr/local/lib/python2.7/site-packages/thumbor/importer.py", line 43, in import_class
    return import_class(name, get_module)
  File "/usr/local/lib/python2.7/site-packages/thumbor/importer.py", line 19, in import_class
    module = get_module and __import__(name) or __import__(module_name)
ImportError: No module named redis_storage

Animated gifs failing with latest 6.0.1 tagged docker image

Requesting /unsafe/300x/http://media.giphy.com/media/fLK0eUlYZoB6E/giphy-tumblr.gif causes a 500 ISE on 6.0.1 but works for 5.2.1e

Logs

2016-05-26 14:45:12 thumbor:ERROR [BaseHandler] get_image failed for url `http%3A//media.giphy.com/media/fLK0eUlYZoB6E/giphy-tumblr.gif`. error: `[Errno 2] No such file or directory`
2016-05-26 14:45:12 tornado.access:ERROR 500 GET /unsafe/300x/http://media.giphy.com/media/fLK0eUlYZoB6E/giphy-tumblr.gif (10.17.225.239) 679.78ms
2016-05-26 14:45:30 thumbor:ERROR ERROR: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 111, in get_image
    self.context.request.image_url
  File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1008, in run
    value = future.result()
  File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 232, in result
    raise_exc_info(self._exc_info)
  File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1017, in run
    yielded = self.gen.send(value)
  File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 535, in _fetch
    fetch_result.buffer = self.context.request.engine.read(extension)
  File "/usr/local/lib/python2.7/site-packages/thumbor/engines/__init__.py", line 51, in read
    extension)
  File "/usr/local/lib/python2.7/site-packages/thumbor/engines/pil.py", line 238, in read_multiple
    popen = Popen(command, stdout=PIPE)
  File "/usr/local/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/local/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

Env vars for docker

sudo docker run -e UPLOAD_ENABLED=True -e DETECTORS=['"thumbor.detectors.feature_detector"'] -p 8003:8000 apsl/thumbor
Quote very important, else you will have error "NameError: name 'thumbor' is not defined"

botocore 1.2.12 not available on pip

Could not find a version that satisfies the requirement botocore==1.2.12 (from -r /usr/src/app/requirements.txt (line 4)) (from versions: 0.4.1, 0.4.2, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.6.0, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.9.0, 0.9.1, 0.9.2, 0.10.0, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.15.0, 0.15.1, 0.16.0, 0.17.0, 0.18.0, 0.19.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.33.0, 0.34.0, 0.35.0, 0.36.0, 0.37.0, 0.38.0, 0.39.0, 0.40.0, 0.41.0, 0.42.0, 0.43.0, 0.44.0, 0.45.0, 0.46.0, 0.47.0, 0.48.0, 0.49.0, 0.50.0, 0.51.0, 0.52.0, 0.53.0, 0.54.0, 0.55.0, 0.56.0, 0.57.0, 0.58.0, 0.59.0, 0.60.0, 0.61.0, 0.62.0, 0.63.0, 0.64.0, 0.65.0, 0.66.0, 0.67.0, 0.68.0, 0.69.0, 0.70.0, 0.71.0, 0.72.0, 0.73.0, 0.74.0, 0.75.0, 0.76.0, 0.77.0, 0.78.0, 0.79.0, 0.80.0, 0.81.0, 0.82.0, 0.83.0, 0.84.0, 0.85.0, 0.86.0, 0.87.0, 0.88.0, 0.89.0, 0.90.0, 0.91.0, 0.92.0, 0.93.0, 0.94.0, 0.95.0, 0.96.0, 0.97.0, 0.98.0, 0.99.0, 0.100.0, 0.101.0, 0.102.0, 0.103.0, 0.104.0, 0.105.0, 0.106.0, 0.107.0, 0.108.0, 0.109.0, 1.0.0a1, 1.0.0a2, 1.0.0a3, 1.0.0b1, 1.0.0b2, 1.0.0b3, 1.0.0rc1, 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.1.9, 1.1.10, 1.1.11, 1.1.12, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.2.11, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.3.8, 1.3.9, 1.3.10, 1.3.11, 1.3.12, 1.3.13, 1.3.14, 1.3.15, 1.3.16, 1.3.17, 1.3.18, 1.3.19, 1.3.20, 1.3.21, 1.3.22, 1.3.23, 1.3.24, 1.3.25, 1.3.26, 1.3.27, 1.3.28, 1.3.29, 1.3.30, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.7, 1.4.8, 1.4.9, 1.4.10, 1.4.11, 1.4.12, 1.4.13, 1.4.14, 1.4.15, 1.4.16, 1.4.17, 1.4.18, 1.4.19, 1.4.20, 1.4.21, 1.4.22, 1.4.23, 1.4.24, 1.4.25, 1.4.26, 1.4.27, 1.4.28, 1.4.29, 1.4.30, 1.4.31, 1.4.32, 1.4.33, 1.4.34, 1.4.35, 1.4.36, 1.4.37, 1.4.38, 1.4.39, 1.4.40, 1.4.41, 1.4.42, 1.4.43, 1.4.44, 1.4.46, 1.4.47, 1.4.48, 1.4.49, 1.4.50, 1.4.51, 1.4.52, 1.4.53, 1.4.54, 1.4.55, 1.4.56, 1.4.57, 1.4.58, 1.4.59, 1.4.60, 1.4.61, 1.4.62, 1.4.63, 1.4.64, 1.4.65)

introduced in 116ff89

Error using docker-compose example

I'm trying to get a basic container running using your examples, but docker-compose if throwing errors.

This is probably a duplicate of #2, but as it's closed I figured best opening a new ticket.

The example I'm using is:

thumbor:
  image: apsl/thumbor:latest
  volumes:
    - data:/data
  ports:
    - "80:8000" # thumbor port
  environment:
    DETECTORS: ['thumbor.detectors.face_detector','thumbor.detectors.feature_detector',]

And the error is:

thumbor.environment.DETECTORS contains ["thumbor.detectors.face_detector", "thumbor.detectors.feature_detector"], which is an invalid type, it should be a string, number, or a null

docker version: 1.11.1, build 5604cbe
docker-compose version: 1.7.1, build 0a9ab35

Trying it on a newly installed, vanilla DigitalOcean Ubuntu 14.04 VPS.

If you're struggling to recreate it, could you let me know what versions of docker/docker compose you're using so I can try with those?

Volume creation does not work

With latest build, logs folder is not creating log file to local volume.
Current docker-compose file contains:
thumbor:
image: apsl/thumbor-multiprocess:5.2.1c
volumes:
- "./data:/data"
- "./logs:/logs"

tc_aws integration seems to have broken thumbor_aws

Now getting:

thumbor_1  | ---> Starting circus...
thumbor_1  | 2016-02-08 12:54:39 circus[1] [INFO] Starting master on pid 1
thumbor_1  | 2016-02-08 12:54:39 circus[1] [INFO] sockets started
thumbor_1  | 2016-02-08 12:54:39 circus[1] [INFO] Arbiter now waiting for commands
thumbor_1  | 2016-02-08 12:54:39 circus[1] [INFO] thumbor started
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | libdc1394 error: Failed to initialize libdc1394
thumbor_1  | 2016-02-08 12:55:02 tornado.application:ERROR Future exception was never retrieved: Traceback (most recent call last):
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1014, in run
thumbor_1  |     yielded = self.gen.throw(*exc_info)
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/imaging.py", line 32, in check_image
thumbor_1  |     exists = yield gen.maybe_future(self.context.modules.storage.exists(kw['image'][:self.context.config.MAX_ID_LENGTH]))
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1008, in run
thumbor_1  |     value = future.result()
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 232, in result
thumbor_1  |     raise_exc_info(self._exc_info)
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 282, in wrapper
thumbor_1  |     yielded = next(result)
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/thumbor/storages/mixed_storage.py", line 86, in exists
thumbor_1  |     result = yield gen.maybe_future(self.file_storage.exists(path))
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/thumbor_aws/storages/s3_storage.py", line 116, in exists
thumbor_1  |     file_key = self.storage.get_key(file_abspath)
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/boto/s3/bucket.py", line 192, in get_key
thumbor_1  |     key, resp = self._get_key_internal(key_name, headers, query_args_l)
thumbor_1  |   File "/usr/local/lib/python2.7/site-packages/boto/s3/bucket.py", line 230, in _get_key_internal
thumbor_1  |     response.status, response.reason, '')
thumbor_1  | S3ResponseError: S3ResponseError: 403 Forbidden

Nothing with my AWS permissions have changed.

passing log level to thumbor

I did not find an easy way to pass the log level to the thumbor command line. I think it should be added because it makes debugging easier?

Example: launching the following command, storage is working but result_storage is not. Not sure what's wrong

docker run -d --restart=always -p 80:80 -v /home/ubuntu/thumbor/data:/data -v /home/ubuntu/thumbor/logs:/logs -e "RESULT_STORAGE=thumbor.result_storages.file_storage" apsl/thumbor

Missing MAX_PIXELS config in thumbor.conf.tpl

I am getting en error when I am trying to process extra large images. Thumbor has the config to change it but APSL/docker-thumbor does not have into its thumbor/conf/thumbor.conf.tpl

I added it here is the PR - #30

Remotecv file loader?

I want to install thumbor as an image resizer / thumbnailer for a website hosted on a local server. I have access to all images locally and I don't need to use http loader for them. I tried the following thumbor settings:

volumes: 
   - "/var/www/fileadmin:/var/srv/fileadmin"
envs:
   - LOADER=thumbor.loaders.file_loader
   - FILE_LOADER_ROOT_PATH=/var/srv

but remotecv doesn't seem to have file_loader mechanism and errors out with image not found...
Is there any benefit in using remotecv in my scenario or should I just rely on thumbor "direct" detectors? Or am I missing a setting in remotecv?

New Pypy Server replacement for wheelhouse cp fails on Docker for Mac

I'm wondering what would this new pypy server need to work on Mac OS X. Docker for Mac exposes everything on localhost. I'm assuming this works fine on Linux. I tried building master and it fails. I also tried hardcoding the IP to point to localhost and that fails as well

Command executed: ./builder

Output using default:

Step 13 : RUN pip install --trusted-host None --no-cache-dir --use-wheel    --extra-index-url http://${DOCKERHOST}:9009/simple/    --trusted-host ${DOCKERHOST}    -r /usr/src/app/requirements.txt
 ---> Running in 5af8b697b983
Collecting argparse==1.2.1 (from -r /usr/src/app/requirements.txt (line 1))
  Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f44a52973d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f44a5297550>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f44a52976d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f44a5297850>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f44a52979d0>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/

Output using 127.0.0.1:

Step 13 : RUN pip install --trusted-host None --no-cache-dir --use-wheel    --extra-index-url http://127.0.0.1:9009/simple/    --trusted-host 127.0.0.1    -r /usr/src/app/requirements.txt
 ---> Running in ee1f66001a6e
Collecting argparse==1.2.1 (from -r /usr/src/app/requirements.txt (line 1))
  Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f1ac2137390>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=3, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f1ac2137510>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=2, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f1ac2137690>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=1, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.HTTPConnection object at 0x7f1ac2137810>: Failed to establish a new connection: [Errno 111] Connection refused',)': /simple/argparse/
  Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<

Graceful monoprocess restart

First of all, sorry for asking this as an issue but I couldn't find anywhere else to ask this kind of question.

So, how can I graceful restart my docker container (and thumbor as consequence) in a way it gets my recently updated environment variables (from a file) before it restarts thumbor and the container itself?

I'm running it in production (in a small project so far) with something like:
docker run --env-file conf/docker-thumbor-vars.env ...

The thing is I want to update some Thumbor configuration variables and cause 0s downtime.

How to build and contribute?

You have a customized build process which is not documented. It would be helpful to share the process you use for development so that others can more easily contribute.

Thanks

Change Remotecv redis envvars

The remotecv redis envvars it's defined like:
REDIS_HOST
REDIS_PORT
REDIS_DATABASE

It will be more explicit if defined like:
REMOTECV_REDIS_HOST
REMOTECV_REDIS_PORT
REMOTECV_REDIS_DATABASE

tc_aws fails load async

I forget to remove a test version for a tc_aws in the requirements file.
It cause problems with async loading.

Enable webp will prevent image cached in result_storage

I noticed that after enable AUTO_WEBP=True then image don't auto save to result_storage folder. Here is my docker-compose.yml config

thumbor:
    image: apsl/thumbor
    volumes:
      - ./stack/thumbor/.data:/data
    environment:
      - ALLOW_UNSAFE_URL=True
      - SECURITY_KEY=Cyberdyne2017
      - RESULT_STORAGE_STORES_UNSAFE=True
      - AUTO_WEBP=True
  nginx:
    image: apsl/thumbor-nginx:latest
    links:
      - thumbor:thumbor
    volumes:
      - ./stack/thumbor/.data:/data
    ports:
      - 12003:80

Not sure if its related to this issue Webp from result_storage is not setting Vary header #731
.

Possible to use newer botocore?

Hi!

We're using AWS ECS to run thumbor. In ECS it is possible to configure Task Role for a container, so AWS SDK inside container can automatically figure out credentials to use, so it won't be necessary to pass credentials via env vars or to put credentials file inside container.
Unfortunately, minimum version of botocore should be 1.4.37.
So, is it possible to upgrade to newer botocore?

dependency not exist botocore==1.2.12

hi
in requirement.txt @ remotecv and thumbor
botocore==1.2.12 not exist

Could not find a version that satisfies the requirement botocore==1.2.12 (from -r /usr/src/app/requirements.txt (line 4)) (from versions: 0.4.1, 0.4.2, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.6.0, 0.7.0, 0.8.0, 0.8.1, 0.8.2, 0.8.3, 0.9.0, 0.9.1, 0.9.2, 0.10.0, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.15.0, 0.15.1, 0.16.0, 0.17.0, 0.18.0, 0.19.0, 0.20.0, 0.21.0, 0.22.0, 0.23.0, 0.24.0, 0.25.0, 0.26.0, 0.27.0, 0.28.0, 0.29.0, 0.30.0, 0.31.0, 0.32.0, 0.33.0, 0.34.0, 0.35.0, 0.36.0, 0.37.0, 0.38.0, 0.39.0, 0.40.0, 0.41.0, 0.42.0, 0.43.0, 0.44.0, 0.45.0, 0.46.0, 0.47.0, 0.48.0, 0.49.0, 0.50.0, 0.51.0, 0.52.0, 0.53.0, 0.54.0, 0.55.0, 0.56.0, 0.57.0, 0.58.0, 0.59.0, 0.60.0, 0.61.0, 0.62.0, 0.63.0, 0.64.0, 0.65.0, 0.66.0, 0.67.0, 0.68.0, 0.69.0, 0.70.0, 0.71.0, 0.72.0, 0.73.0, 0.74.0, 0.75.0, 0.76.0, 0.77.0, 0.78.0, 0.79.0, 0.80.0, 0.81.0, 0.82.0, 0.83.0, 0.84.0, 0.85.0, 0.86.0, 0.87.0, 0.88.0, 0.89.0, 0.90.0, 0.91.0, 0.92.0, 0.93.0, 0.94.0, 0.95.0, 0.96.0, 0.97.0, 0.98.0, 0.99.0, 0.100.0, 0.101.0, 0.102.0, 0.103.0, 0.104.0, 0.105.0, 0.106.0, 0.107.0, 0.108.0, 0.109.0, 1.0.0a1, 1.0.0a2, 1.0.0a3, 1.0.0b1, 1.0.0b2, 1.0.0b3, 1.0.0rc1, 1.0.0, 1.0.1, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.1.9, 1.1.10, 1.1.11, 1.1.12, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.2.8, 1.2.9, 1.2.10, 1.2.11, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.3.8, 1.3.9, 1.3.10, 1.3.11, 1.3.12, 1.3.13, 1.3.14, 1.3.15, 1.3.16, 1.3.17, 1.3.18, 1.3.19, 1.3.20, 1.3.21, 1.3.22, 1.3.23, 1.3.24, 1.3.25, 1.3.26, 1.3.27, 1.3.28, 1.3.29, 1.3.30, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.7, 1.4.8, 1.4.9, 1.4.10, 1.4.11, 1.4.12, 1.4.13, 1.4.14, 1.4.15, 1.4.16, 1.4.17, 1.4.18, 1.4.19, 1.4.20, 1.4.21, 1.4.22, 1.4.23, 1.4.24, 1.4.25, 1.4.26, 1.4.27, 1.4.28, 1.4.29, 1.4.30, 1.4.31, 1.4.32, 1.4.33, 1.4.34, 1.4.35, 1.4.36, 1.4.37, 1.4.38, 1.4.39, 1.4.40, 1.4.41, 1.4.42, 1.4.43, 1.4.44, 1.4.46, 1.4.47, 1.4.48, 1.4.49, 1.4.50, 1.4.51, 1.4.52, 1.4.53, 1.4.54, 1.4.55, 1.4.56, 1.4.57, 1.4.58, 1.4.59, 1.4.60, 1.4.61, 1.4.62, 1.4.63, 1.4.64, 1.4.65, 1.4.66, 1.4.67, 1.4.68, 1.4.69, 1.4.70, 1.4.71, 1.4.72, 1.4.73, 1.4.74, 1.4.75, 1.4.76, 1.4.77, 1.4.78)
No matching distribution found for botocore==1.2.12 (from -r /usr/src/app/requirements.txt (line 4))

NameError: name 'true' is not defined

Hey there,
first thanks, for this docker.
I have following docker-compose

images:
  image: apsl/thumbor
  stdin_open: true
  tty: true
  restart: always
  environment:
    MAX_AGE: 86400
    MIN_WIDTH: 100
    MIN_HEIGHT: 100
    ALLOW_ANIMATED_GIFS: True
    AUTO_WEBP: True
    SECURITY_KEY: "LKJSAKDnmkwle2wewkl242NKnjs230kljknm"
    ALLOW_UNSAFE_URL: False
    ALLOW_OLD_URLS: False
    STORAGE_EXPIRATION_SECONDS: 7776000 #90 days
    FILE_STORAGE_ROOT_PATH: '/data/storage'
    RESULT_STORAGE_EXPIRATION_SECONDS: 2592000 #30 days
    UPLOAD_MAX_SIZE: 10240 # 10MB
    UPLOAD_ENABLED: True
    UPLOAD_PUT_ALLOWED: True
  volumes:
    - /images/:/data/storage/:rw

When I try to start it I get following error repentantly thrown.

11/11/2015 2:46:42 AMTraceback (most recent call last):
11/11/2015 2:46:42 AM  File "/usr/local/bin/thumbor", line 9, in <module>
11/11/2015 2:46:42 AM    load_entry_point('thumbor==5.2.1', 'console_scripts', 'thumbor')()
11/11/2015 2:46:42 AM  File "/usr/local/lib/python2.7/site-packages/thumbor/server.py", line 46, in main
11/11/2015 2:46:42 AM    config = Config.load(server_parameters.config_path, conf_name='thumbor.conf', lookup_paths=lookup_paths)
11/11/2015 2:46:42 AM  File "/usr/local/lib/python2.7/site-packages/derpconf/config.py", line 75, in load
11/11/2015 2:46:42 AM    return cls.__load_from_path(conf, path)
11/11/2015 2:46:42 AM  File "/usr/local/lib/python2.7/site-packages/derpconf/config.py", line 84, in __load_from_path
11/11/2015 2:46:42 AM    six.exec_(code, module.__dict__)
11/11/2015 2:46:42 AM  File "/usr/local/lib/python2.7/site-packages/six.py", line 699, in exec_
11/11/2015 2:46:42 AM    exec("""exec _code_ in _globs_, _locs_""")
11/11/2015 2:46:42 AM  File "<string>", line 1, in <module>
11/11/2015 2:46:42 AM  File "<string>", line 50, in <module>
11/11/2015 2:46:42 AMNameError: name 'true' is not defined

Hope you can help. : )

"ValueError: unrecognized raw mode" in GIF

Hi,
if I try to resize an animated gif using the thumborize.me service the image is resized perfectly
http://thumbor.thumborize.me/unsafe/150x100/https://secure.static.tumblr.com/b2a7fc9c77aed0ede4c1abd070872f96/vsbwkai/gZEnpmdmk/tumblr_static_tumblr_static_6wanbc98p1ssco8cw88oks0wc_640.gif

If I try to resize the image using my docker-thumbor (v 6.1.5) I get the following error

nginx_1     | 79.16.57.198 [10/Nov/2016:15:01:31 +0000] "GET /unsafe/100x100/https://secure.static.tumblr.com/b2a7fc9c77aed0ede4c1abd070872f96/vsbwkai/gZEnpmdmk/tumblr_static_tumblr_static_6wanbc98p1ssco8cw88oks0wc_640.gif HTTP/1.1" 500 0 "-" 0.220  0.220
thumbor_1   | 2016-11-10 15:01:31 thumbor:ERROR ERROR: Traceback (most recent call last):
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 142, in get_image
thumbor_1   |     self.context.request.image_url
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1015, in run
thumbor_1   |     value = future.result()
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 237, in result
thumbor_1   |     raise_exc_info(self._exc_info)
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1024, in run
thumbor_1   |     yielded = self.gen.send(value)
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 575, in _fetch
thumbor_1   |     fetch_result.buffer = self.context.request.engine.read(extension)
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/thumbor/engines/__init__.py", line 75, in read
thumbor_1   |     extension)
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/thumbor/engines/pil.py", line 227, in read_multiple
thumbor_1   |     converted_images.append(im.convert("RGB"))
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/PIL/Image.py", line 841, in convert
thumbor_1   |     self.load()
thumbor_1   |   File "/usr/local/lib/python2.7/site-packages/PIL/Image.py", line 760, in load
thumbor_1   |     self.im.putpalette(*self.palette.getdata())
thumbor_1   | ValueError: unrecognized raw mode

Can you help me please to understand where I'm wrong?

Newrelic support

Do you have any plan to add newrelic support?
So it installs newrelic-sysmond and accepts license key via env variables.

Issue with docker-compose

Hello,

I have been trying to use docker-compose to stand up a Thumbor server but I continue to get the following error:

ERROR: Validation failed, reason(s):
Service 'thumbor' configuration key 'environment' 'DETECTORS' contains ['thumbor.detectors.feature_detector', 'thumbor.detectors.face_detector'], which is an invalid type, it should be a string, number, boolean or a null

My docker-compose.yml file is a slightly modified version of the docker-compose-production.yml except I have added in the S3 configuration stuff.

thumbor:
  image: apsl/thumbor-multiprocess:latest
  volumes:
    - data:/data
    - logs:/logs
  #ports:
    #- "80:80" # thumbor port
  environment:
    ALLOW_UNSAFE_URL: 'False'
    SECURITY_KEY: ...
    DETECTORS: ['thumbor.detectors.feature_detector','thumbor.detectors.face_detector',]
    AWS_ACCESS_KEY: ...
    AWS_SECRET_KEY: ...
    STORAGE: thumbor.storages.mixed_storage
    STORAGE_BUCKET: ...
    RESULT_STORAGE_BUCKET: ...-result
    UPLOAD_PHOTO_STORAGE: thumbor_aws.storages.s3_storage
    MIXED_STORAGE_FILE_STORAGE: thumbor_aws.storages.s3_storage
    RESULT_STORAGE: thumbor_aws.result_storages.s3_storage
    REDIS_STORAGE_SERVER_HOST: redis
    REDIS_STORAGE_SERVER_PORT: 6379
    REDIS_STORAGE_SERVER_DB: 0
    REDIS_QUEUE_SERVER_HOST: redis
    REDIS_QUEUE_SERVER_PORT: 6379
    REDIS_QUEUE_SERVER_DB: 0
    MIXED_STORAGE_DETECTOR_STORAGE: thumbor.storages.redis_storage
    #SENTRY_DSN_URL: # put your sentry enpoint here
    THUMBOR_NUM_PROCESSES: 16
  links:
    - redis:redis
nginx:
  image: apsl/thumbor-nginx:latest
  links:
    - thumbor:thumbor
  volumes_from:
    - thumbor
  ports:
    - "80:80" # nginx cache port (with failover to thumbor)
  hostname: ...
remotecv:
  image: apsl/remotecv:latest
  links:
    - redis:redis
  environment:
    REDIS_HOST: redis
    REDIS_PORT: 6379
    REDIS_DATABASE: 0
redis:
  image: redis:latest

I am doing a simple docker-compose up. I am probably just missing something. Can you point me in the right direction?

Thanks!

Remotecv fails when activate sentry

remotecv_1       | DEBUG:raven.base.Client:Configuring Raven for host: <raven.conf.remote.RemoteConfig object at 0x7fa9bb1c4e50>
remotecv_1       | ERROR:sentry.errors.serializer:'Requirement' object has no attribute 'name'
remotecv_1       | Traceback (most recent call last):
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/raven/utils/serializer/manager.py", line 76, in transform
remotecv_1       |     return repr(value)
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2829, in __repr__
remotecv_1       |     def __repr__(self): return "Requirement.parse(%r)" % str(self)
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/pkg_resources/_vendor/packaging/requirements.py", line 110, in __str__
remotecv_1       |     parts = [self.name]
remotecv_1       | AttributeError: 'Requirement' object has no attribute 'name'
remotecv_1       | DEBUG:raven.base.Client:Sending message of length 2842 to http://sentry3.apsl.net/api/10/store/
remotecv_1       | Traceback (most recent call last):
remotecv_1       |   File "/usr/local/bin/remotecv", line 11, in <module>
remotecv_1       |     sys.exit(main())
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/remotecv/worker.py", line 95, in main
remotecv_1       |     config.error_handler = ErrorHandler(arguments.sentry_url)
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/remotecv/error_handler.py", line 17, in __init__
remotecv_1       |     'modules': self.get_modules()
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/remotecv/error_handler.py", line 26, in get_modules
remotecv_1       |     res_mod = pkg_resources.get_distribution(module)
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 533, in get_distribution
remotecv_1       |     dist = Requirement.parse(dist)
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2833, in parse
remotecv_1       |     req, = parse_requirements(s)
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2781, in parse_requirements
remotecv_1       |     yield Requirement(line)
remotecv_1       |   File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2790, in __init__
remotecv_1       |     raise RequirementParseError(str(e))
remotecv_1       | pkg_resources.RequirementParseError: Invalid requirement, parse error at "'_LWPCook'"

Size marking

Hi!

I am using - https://github.com/APSL/docker-thumbor. Is there a way to configure Thumbor to mark size of image upon resize request? e.g.:

http://thumbor:9000/unsafe/100x/s3-eu-central-1.amazonaws.com/grok7-dev/orionjs/15d0654f-88d7-4b31-ad5f-7ffb434e9d1b.jpg

Result:

https://s3-eu-central-1.amazonaws.com/thumbor/grok7-dev/orionjs/100x/15d0654f-88d7-4b31-ad5f-7ffb434e9d1b.jpg

Or

https://s3-eu-central-1.amazonaws.com/thumbor/grok7-dev/orionjs/15d0654f-88d7-4b31-ad5f-7ffb434e9d1bx100.jpg

Using Docker-Thumbor with AWS Elastic Beanstalk

Hello, and sorry if this is a stupid isse :-)

We have user images uploaded to S3 and want to resize them dynamically using docker-thumbor deployed to an EC2 instance.

For this we deployed the Dockerrun.aws.json to AWS Elastic Beanstalk to dynamically resize the user profile images stored in an S3 bucket:

{ "AWSEBDockerrunVersion": "1", "Image": { "Name": "apsl/thumbor" }, "Ports": [ { "ContainerPort": "8000" } ] }

The Thumbor Docker image is pulled by the Beanstalk instance from https://hub.docker.com/r/apsl/thumbor/ by AWS, and we give it the TC_AWS_LOADER_BUCKET='foocompany-images' environment variable, where foocompany-images is the name of the S3 bucket.

I expected the url .../unsafe/100x100/foocompany-images/existingimage.jpg to work with this setting, but it always gives a 404. Unfortunately the instance logs don't give any indication that Thumbor tries to access S3 or that the TC_AWS_ setting is applied at all.

Our Thumbor docker container works with the example URL where an image is fetched and resized from imgur.

Could it be that TC_AWS_XXX is not activated in the base image?

Graphicsmagick engine fails

We need to include more dependencies:
graphicsmagick libgraphicsmagick++3 libgraphicsmagick3 libgraphicsmagick++1-dev libgraphicsmagick1-dev python-dev libboost-python-dev

And a new pip package pgmagick

Can't load images using `tc_aws.loaders.s3_loader`

I'm using https://github.com/APSL/docker-thumbor with tc_aws.loaders.s3_loader, but I can't seem to get it to work. The error I'm getting is:

2016-11-22 09:26:57 thumbor:ERROR [BaseHandler] get_image failed for url `bucket-name/design-example-1.jpg`. error: `Parameter validation failed:
Invalid bucket name "": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$"`
2016-11-22 09:26:57 tornado.access:ERROR 500 GET /PbhAZkbZdM0Q5AWPztSfaFdalNY=/fit-in/500x500/bucket-name/design-example-1.jpg (172.31.1.220) 39.45ms

which seems to imply that it's failing to extract the bucket name from my url... is the url I'm generating wrong or is this a bug/configuration problem?

This is the configuration I'm using:

MAX_WIDTH=2000
MAX_HEIGHT=2000
SECURITY_KEY='****'
ALLOW_UNSAFE_URL=False
LOADER='tc_aws.loaders.s3_loader'
STORAGE='thumbor.storages.no_storage'
RESULT_STORAGE='tc_aws.result_storages.s3_storage'
AWS_ACCESS_KEY_ID='****'
AWS_SECRET_ACCESS_KEY='****'
TC_AWS_REGION='eu-west-1'
TC_AWS_RESULT_STORAGE_BUCKET='another-bucket-name'
TC_AWS_RESULT_STORAGE_ROOT_PATH=''
TC_AWS_MAX_RETRY=3
TC_AWS_STORAGE_SSE=False
TC_AWS_STORAGE_RRS=True
TC_AWS_ENABLE_HTTP_LOADER=False
TC_AWS_ALLOWED_BUCKETS=False
TC_AWS_STORE_METADATA=False

Thanks

Ability to extend config.tpl

Would be nice to have an optional file which will extend current thumbor.conf.tpl - e.g. providing new config parameters. That's useful in case when using thumbor-plugins which define more config options. Currently I have to copy whole thumbor.conf.tpl, add new parameters to the end and ADD it in my Dockerfile. That works well, but with new releases of docker images I have to go back to this repo and grab latest conf file and reinsert my custom parameters again.

Warning message about favicon.ico

Hi,
I'm keep having these error messages at each request

tornado.access:WARNING 400 GET /favicon.ico (192.168.99.1) 2.51ms
thumbor:WARNING URL does not have hash or unsafe, or has both: /favicon.ico

Any idea ?

Cannot store image results on S3

Hi!

I'm having some issues with my result storage, it was setup to use S3 but for some reason Thumbor is not creating any post processed images on it, however the other storage for original images seems to be working just fine.

Is there something wrong with my docker-compose configuration? Here it follows:

  thumbor:
    image: apsl/thumbor:6.0.1
    restart: always
    ports:
      - "8888:8000" # thumbor port
    environment:
      SECURITY_KEY: ".............."
      VIRTUAL_HOST: thumbor.docker
      STORAGE: "tc_aws.storages.s3_storage"
      RESULT_STORAGE: "tc_aws.result_storages.s3_storage"
      TC_AWS_STORAGE_BUCKET: "test"
      TC_AWS_STORAGE_ROOT_PATH: "/thumbor/storage"
      TC_AWS_RESULT_STORAGE_BUCKET: "test"
      TC_AWS_RESULT_STORAGE_ROOT_PATH: "/thumbor/result_storage"
      AWS_ACCESS_KEY_ID: ".............."
      AWS_SECRET_ACCESS_KEY: ".............."

Thumbor Version 6.3

Thumbor has recently released version 6.3. Would it be possible to bump the Thumbor version? Thanks.

S3 results storage fails with smart detector

When using S3 results storage, requests fail with smart detectors
It may well be the same issue as this:
thumbor/thumbor#721
thumbor/thumbor#722

Error:

Exception: DummyFuture does not support blocking for results
2016-10-08 19:44:39 thumbor:WARNING [AwsStorage] s3 key not found at images/zwhvs1hkqj57.detectors.txt
2016-10-08 19:45:09 tornado.application:ERROR Future exception was never retrieved: Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/tornado/gen.py", line 1024, in run
    yielded = self.gen.send(value)
  File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 211, in get_image
    self.filters_runner.apply_filters(thumbor.filters.PHASE_AFTER_LOAD, transform)
  File "/usr/local/lib/python2.7/site-packages/thumbor/filters/__init__.py", line 81, in apply_filters
    callback()
  File "/usr/local/lib/python2.7/site-packages/thumbor/handlers/__init__.py", line 209, in transform
    self.context.transformer.transform(after_transform_cb)
  File "/usr/local/lib/python2.7/site-packages/thumbor/transformer.py", line 102, in transform
    self.smart_detect()
  File "/usr/local/lib/python2.7/site-packages/thumbor/transformer.py", line 151, in smart_detect
    self.do_smart_detection().result()
  File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 238, in result
    self._check_done()
  File "/usr/local/lib/python2.7/site-packages/tornado/concurrent.py", line 316, in _check_done
    raise Exception("DummyFuture does not support blocking for results")
Exception: DummyFuture does not support blocking for results

Env Config:

              environment:
                - LOADER=tc_aws.loaders.s3_loader
                - STORAGE=tc_aws.storages.s3_storage
                - DETECTORS=['thumbor.detectors.feature_detector','thumbor.detectors.face_detector']
                - AWS_ACCESS_KEY_ID=XXXXXXX
                - AWS_SECRET_ACCESS_KEY=YYYYYYYY
                - TC_AWS_REGION=us-west-2
                - TC_AWS_STORAGE_BUCKET=my-bucket
                - TC_AWS_LOADER_BUCKET=my-bucket
                - TC_AWS_RESULT_STORAGE_BUCKET=my-bucket-results
                - TC_AWS_ALLOWED_BUCKETS=['my-bucket','my-bucket-results']
                - RESULT_STORAGE=tc_aws.result_storages.s3_storage

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.