codesuki / ecs-nginx-proxy Goto Github PK

If you run your tasks without specifying the host port mapping, they get a random port assigned.
Should you run say the same task on 3 machines, but the ecs proxy on only one machine, the ecs proxy will fail to redirect traffic to your instances because the security groups won't have the random port open by default.

Let me know your thoughts, but I think task containers need to have a fixed host port (unless I missed something? )

https://github.com/codesuki/ecs-nginx-proxy/blob/master/examples/sample_task.json#L8

Don't know if helpful, but to redirect http to https, I used this code:
https://stackoverflow.com/a/35097290/3019499

server {
  listen 81;
  return 301 https://$host$request_uri;
}

Then the ELB listens on port 80 and goes to port 81 for the ecs-nginx-proxy.

Not sure if this should be a default handy feature of the proxy, or turned on on demand

I'm trying to set up the nginx proxy on my Amazon AWS Docker instance together with a pimcore instance. This is my compose file:

version: '3.4'

services:  
  nginx-proxy:
    image: codesuki/ecs-nginx-proxy
    ports:
      - "80:80"
  pimcore-jcii:
    image: ****/pimcore5:current
    ports:
      - "8000:80"
    links:
      - "db"
    volumes:
      - efs-storage:/data
  db:
    image: mariadb
    restart: always
    ports:
      - "3306:3306"

volumes:
  efs-storage:
    driver: cloudstor:aws
    driver_opts:
      backing: shared

If I deploy this stack the nginx proxy container can not start. It appears the following error message:
task: non-zero exit (1)

What am I doing wrong?

After the earlier commit's #14 #15, I started adding some of the nginx properties via environment variables. It works fine when you have one container running under nginx proxy but when you have multiple containers, values are getting duplicated.

example:

container -1:

key: NGINX_GEN_keepalive_timeout value: 120s

container -2:

key: NGINX_GEN_keepalive_timeout value: 120s

when ecs-gen writes the nginx.conf file it's adding same value twice.

server {

    server_name xxxxxxxxxxx;

    keepalive_timeout 120s;

    keepalive_timeout 120s;

    listen 80;

    access_log /var/log/nginx/access.log vhost;

    location / {

            proxy_pass http://xxxxxxxxxxx;

    }

}

How can we avoid duplication and have only one key value? I am not familiar with go html template so any help is appreciated.

@codesuki @smaclell

Thanks for creating this project.

I am using ecs-nginx-proxy deployed in our dev environment ecs cluster with about 77 tasks running. Intermittently, various users report getting the nginx 503 temporarily unavailable error message.

I have been using a load test with 100 clients hitting container url's over a 15 second period in order to reliably reproduce this and dig into some logs. While I was troubleshooting, I would periodically check the AWS console page for ECS to look at one the task definition for the proxy, and got "Failed to describe private-ecs-nginx-proxy - Rate exceeded".

While examining the code for ecs-gen, it became clear that describeTaskDefinition is being used for each request in order to route to the correct container (because that is where you would find the VIRTUAL_HOST env var, of course). But it seems that if the api returns an error, the code returns nil and there is nothing being logged. I am wondering if:

a) have you ever run into this rate limit before in your usage or testing?
b) do you have any plans to produce some logging for troubleshooting in ecs-gen :) ?
c) Can ecs-gen perhaps cache the list of task definitions for a short period of time so that we can avoid hitting this limit?

It's a bit annoying that aws does not publish these rate limits, so I realize it's hard to guage how long to wait before invalidating the cache. Perhaps it can be a config variable.

Many thanks,
Julie

It seems we only have http support for now with this image.

nginx-proxy is using a VIRTUAL_PROTO environment variable https://github.com/jwilder/nginx-proxy/blob/985c46d8b56f2ee4652e9d75b49c128c14bee328/nginx.tmpl#L134

And sets a bunch of configs in https://github.com/jwilder/nginx-proxy/blob/985c46d8b56f2ee4652e9d75b49c128c14bee328/nginx.tmpl

I personally have no ideas what these config mean and how to properly configure nginx so I won't be able to propose a PR, but I believe cherry picking his code would be enough.

Also, would the ELB forward port 443 to port 80 of nginx or to port 443?

Running the ecs-nginx-proxy without any special config, I'm getting the following error as soon as I introduce a service:

2017/03/06 03:44:09 found cluster name to be: test-reverse-proxy
2017/03/06 03:44:09 updating config
2017/03/06 03:44:10 container is own container. skipping
2017/03/06 03:44:10 running signal command
2017/03/06 03:44:10 ===== output start =====
2017/03/06 03:44:10 2017/03/06 03:44:10 [emerg] 14#14: could not build server_names_hash, you should increase server_names_hash_bucket_size: 64
nginx: [emerg] could not build server_names_hash, you should increase server_names_hash_bucket_size: 64

2017/03/06 03:44:10 ===== output end =====
2017/03/06 03:44:10 failed to run signal command
2017/03/06 03:44:10 error:  exit status 1
2017/03/06 03:44:10 []

Any idea what could be wrong?

The task definition for the container is:

{
  "requiresAttributes": [
    {
      "value": null,
      "name": "com.amazonaws.ecs.capability.logging-driver.json-file",
      "targetId": null,
      "targetType": null
    },
    {
      "value": null,
      "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19",
      "targetId": null,
      "targetType": null
    }
  ],
  "taskDefinitionArn": "arn:aws:ecs:ap-southeast-2:1234567890:task-definition/hello-world:1",
  "networkMode": "bridge",
  "status": "ACTIVE",
  "revision": 1,
  "taskRoleArn": null,
  "containerDefinitions": [
    {
      "volumesFrom": [],
      "memory": 128,
      "extraHosts": null,
      "dnsServers": null,
      "disableNetworking": null,
      "dnsSearchDomains": null,
      "portMappings": [
        {
          "hostPort": 81,
          "containerPort": 80,
          "protocol": "tcp"
        }
      ],
      "hostname": null,
      "essential": true,
      "entryPoint": null,
      "mountPoints": [],
      "name": "hello-world",
      "ulimits": null,
      "dockerSecurityOptions": null,
      "environment": [
        {
          "name": "VIRTUAL_HOST",
          "value": "helloworld.test-reverse-proxy.a.little.bit.longer"
        }
      ],
      "links": null,
      "workingDirectory": null,
      "readonlyRootFilesystem": null,
      "image": "tutum/hello-world",
      "command": null,
      "user": null,
      "dockerLabels": null,
      "logConfiguration": {
        "logDriver": "json-file",
        "options": {
          "max-size": "200m"
        }
      },
      "cpu": 10,
      "privileged": null,
      "memoryReservation": null
    }
  ],
  "placementConstraints": [],
  "volumes": [],
  "family": "hello-world"
}

Would this ECS nginx proxy work with Fargate too? I am trying to implement it on Fargate but I'm running into problems getting it started

I had this problem when trying ecs-nginx-proxy, how can I get more debug info?

I saw this when I use docker logs.

2017/02/21 01:28:01 found cluster name to be: kanto
2017/02/21 01:28:01 updating config
panic: runtime error: index out of range


goroutine 1 [running]:
panic(0x97b5c0, 0xc420010070)
	/usr/local/go/src/runtime/panic.go:500 +0x1a1
main.(*ecsClient).describeCluster(0xc420020248, 0xc420146100, 0x5, 0x9161e0, 0xc420146230, 0xc4200a5b40)
	/go/src/github.com/codesuki/ecs-gen/ecs.go:27 +0x142
main.(*scanner).scan(0xc420159d80, 0xc420159dc0, 0x412bce, 0xc420020248, 0x8, 0x8)
	/go/src/github.com/codesuki/ecs-gen/scanner.go:29 +0xc3
main.updateAndWrite(0xc420020230, 0xc420020248)
	/go/src/github.com/codesuki/ecs-gen/main.go:95 +0x99
main.execute(0xc420020230, 0xc420020248)
	/go/src/github.com/codesuki/ecs-gen/main.go:80 +0x55
main.main()
	/go/src/github.com/codesuki/ecs-gen/main.go:73 +0x1d1

I've been using nginx-proxy for a while now, and came across this when wanting to move off vanilla ec2 and over to ecs.
I've got it to a point where if I go directly to one of the 2 instances I am creating when I create a cluster (not both), I get the 'nginx is working' page. I have been writing down my understanding as following along, so will paste here:

First make a cluster

The name comes from the ecs config when you run ecs-cli configure...

ecs-cli up --keypair cluster-pair --capability-iam --size 2 --instance-type t2.small --force

Then we register the task that handles the proxy

aws ecs register-task-definition --cli-input-json file://./examples/task.json

Next we create a service, and connect up to the load balancer

(You need to have already created a load balancer)

aws ecs create-service --cluster ClusterName --role ecsServiceRole --load-balancers loadBalancerName=ECSLB,containerName=ecs-nginx-proxy,containerPort=80 --cli-input-json file://./examples/service.json

This tells the service that it should be accessible on port 80, the container is the ecs-nginx-proxy container, and the details are in service.json.

However when I configure the load balancer, and point it at my two instances that were created due to the above, going to its DNS address I get nothing back. Its only if I go directly to one of the instances.

Am I right in thinking that both sample_task and task are running on both instances and the load balancer is supposed to point to both instances?
Thanks

I'm getting the following running ecs-gen on a container.

2017/10/12 04:36:23 found cluster region to be: ap-southeast-2
2017/10/12 04:36:23 found cluster name to be: Test-ECS-Cluster
2017/10/12 04:36:23 updating config
2017/10/12 04:38:59 NoCredentialProviders: no valid providers in chain. Deprecated.