gliderlabs / connectable Goto Github PK
View Code? Open in Web Editor NEWMagic proxy for internal services
License: MIT License
Magic proxy for internal services
License: MIT License
For backends like etcd, typing in the backend name over and over again is tiresome and errorprone. Additionally, why should the container have to specify the servername?
So, if the backend environment variable starts with etcd:// or consul/etc, then it does the same thing that it does now. Otherwise, if its just a raw string, and --default-backend is set, then it appends the env variable to the key and does the lookup that way.
There is a memory leak in this app, I haven't been able to figure out where yet, but after several hours of operations at 10s of thousands if not 100s of thousands of connections, connectable just continues to consume memory.
I'm trying to track it down.
I've been working on building out a CoreOS cluster using ambassadord, and I ran into an error connecting to an etcd backend.
On the command:
docker pull progrium/ambassadord
docker run --name my-app-amb.1 progrium/ambassadord etcd://127.0.0.1:4001/services/my-app
I get the error:
2014/08/26 15:35:04 unrecognized config store backend: etcd
Looking in this repository, I see a few things that lead me to believe the build on docker hub is out of date:
stage/
All of this has me thinking perhaps stage/
needs to be updated via make
to refresh the bits on Docker hub. Does this seem right?
EDIT: Additonally, the hub page lists June 30 as its latest date.
Hey,
I'm wanting to use the consul service catalog with registrator not the key value store because of health checking and DNS goodness.
So I would run ambassadord with BACKEND_8080=httptest.services.consul
to route to a service.
The really cool thing about the configBackends is the .WatchToUpdate()
method because it caches the lookups. The DNS is good for database connections but for lots of HTTP its better to have resolved routes in memory.
A useful thing would be an in memory cache that changes with the service catalog not the key/value store.
I'm thinking there could be a 4th type of backend - like a srvBackends
but that has a .WatchToUpdate()
method listening to /v1/catalog/service/<service>
.
This way the same consul:
registrator path is used for both but the user can choose between current srvBackends routing (for things like long-lived database connections) and cached service routing (for things like HTTP connections).
it could be: BACKEND_8080=/httptest.services.consul
(or something similar) to denote that the service cache mode should be used
Do you think this is useful? I'm happy to make a pull request if you do :)
We ran into #13 while using the latest Image pulled from Dockerhub. It looks like the latest code didn't make it into stage/ambassadord
yet.
I tried to build it myself but my lacking Go karate together with our corporate proxy foiled that attempt. :(
Would you mind building the binary? Or give me a short rundown on how to gather all dependencies for building it?
Can you provide more run examples? How can I use connectable w/o Resolvable?
I can get omni mode working with my ambassadord container named backends.
However if I change the name to something else, like "myservices", the --setup-iptables container fails (even when I change the name of the linked container to match "myservices").
Reverting my container name to "backends" makes the --setup-iptables container work again.
I took a look at the code and it is not obvious to me where the "backends" container name becomes important. Some other names work, like "ba", but not "router" or "routes".
I'm also using fig, so my container names are not even directly backends, but rather "mydir_backends_1".
Is there something I'm missing?
Thanks
connectable is presently failing during the iptables command:
$ docker logs connectable
2015/08/07 18:28:50 Connectable listening on 10000 ...
Self: 2011f14d0013ed337acf696b3def8737b45eb7d787b4de79e34b1294b8781e9d bridge
2015/08/07 18:28:51 non-zero exit: 1
$ docker logs thirsty_mcclintock
2015/08/07 18:28:51 listen tcp :10000: bind: address already in use
If I run docker run -d --name some-mongo --expose 27017 progrium/ambassadord mongo-host:27017
port 10000
is proxied to mongo-host:27017
, but port 27017
of this container is not.
README.md
says "Ambassador to fixed/resolved backend(s) using domain or IP and port. Listens on same port as destination."
Given the following fig.yml
:
app:
build: .
command: python run.py
ports:
- "5000"
environment:
SERVICE_NAME: app
consul:
image: progrium/consul
ports:
- "8400:8400"
- "8500:8500"
- "8600:53/udp"
hostname: node1
command: -server -bootstrap
registrator:
image: progrium/registrator
command: "consul://localdocker:8500"
volumes:
- /var/run/:/tmp/
environment:
SERVICE_NAME: registrator
ambassadord:
image: progrium/ambassadord
ports:
- "80:10000"
command: "app.service.consul"
I get the following problem with ambassadord when accessing http://localdocker/:
fig up ambassadord
Creating samantha_ambassadord_1...
Attaching to samantha_ambassadord_1
ambassadord_1 | 2014/10/08 23:37:11 Ambassadord listening on 10000 using app.service.consul ...
ambassadord_1 | 2014/10/08 23:37:15 dns: lookup app.service.consul on [10.0.2.3]:53: server misbehaving
ambassadord_1 | 2014/10/08 23:37:15 No backends! Closing the connection
Any clue would be excellent.
Docker images from gliderlabs/connectable is broken .
Has to point to gliderlabs/connectable:master
Cheers
Kev
I have containers launched that are:
And I consistently cannot reach the backends.
from backends container logs I see:
backends_1 | 2015/03/23 03:19:57 omni: unable to list containers
backends_1 | 2015/03/23 03:19:57 No backends! Closing the connection
Am I doing it wrong?
using docker 1.5 on osx+virtualbox:
docker@dev:$ docker version$
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.4.1
Git commit (client): a8a31ef
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.4.1
Git commit (server): a8a31ef
docker@dev:
Thanks!
Tyson
Thank you for a such a useful container. This is more of a how-to question. I have connected my mysql
container to the ambassadord
container through the ominmode
option, with registrator
and consul
docker run -v /var/run/docker.sock:/var/run/docker.sock --name backends -d progrium/ambassadord --omnimode
docker run --rm --privileged --net container:backends progrium/ambassadord --setup-iptables
docker run -p 9990:3306 -e SERVICE_NAME=mysql -e BACKEND_3306=mysql.service.consul -e MYSQL_USER=admin -e MYSQL_PASS=PassworD1 --link backends:backends -d tutum/mysql:5.6
Assuming the Private IP address for mysql
host is 172.20.20.10
. I have a 3-cluster consul
servers, at 172.20.20.10
,172.20.20.11
, and 172.20.20.12
.
How do I connect to the mysql
instance from another host 172.20.20.11
? I understand that I will be running another instance of ambassadord
that connects to mysql
amb or would I connecting to the consul
server? Can you help me with the actual command that I should run in the other host to link to the mysql
container. I am running the following, but it's not giving me the correct env variables set at the other host.
docker run --expose 3306 -e MYSQL_PORT_3306_TCP=tcp://172.20.20.10:3306 --name amb -d progrium/ambassadord --links
I wished there was a way to just register the external mysql host through consul services, something like:
docker run --expose 3306 -e backend_3306=mysql.service.consul --name amb -d progrium/ambassadord --links
Im setting up fleet services in CoreOS. I'm using registrator + ambassadord to set up service discovery across multiple hosts.
From what I can tell I'm stuck trying to get a container to recognize whats in etcd.
Starting Registrator
docker run --rm --name registrator -v /var/run/docker.sock:/tmp/docker.sock progrium/registrator -ip=${COREOS_PRIVATE_IPV4} etcd://${COREOS_PRIVATE_IPV4}:4001/services
Starting Ambassadord
docker run --rm --name backends -v /var/run/docker.sock:/var/run/docker.sock progrium/ambassadord --omnimode
Starting Redis
docker run --name redis -p 6379:6379 -e SERVICE_NAME=redis dockerfile/redis
etcd status after registrator
$ etcdctl get /services/redis/core-01:redis:6379
$ 172.17.8.102:6739
Trying to PING the Redis container on a different host
docker run -it --rm --link backends:redis -e BACKEND_6379="etcd://${COREOS_PRIVATE_IPV4}/services/redis" dockerfile/redis bash -c 'redis-cli ping'
Result
Could not connect to Redis at 127.0.0.1:6379: Connection refused
The last docker run errors because its trying to connect to redis on localhost. Am I missing something? Do I have to build a container that supports ambassadord?
Hi
I have two CoreOS clusters, one on Auzre and one on AWS.
They both run registrator, skydns. I am running a SDN (Weave).
I would like to be able to transparently discover services across clouds.
My idea is to use Ambassadord in omni mode, on each cloud, which has an IP on the SDN and can communicate with the Ambassadors on other clouds (and possibly DNS)
See diagram:https://dl.dropboxusercontent.com/u/31838796/MultiCloud-Networking%20and%20Discovery.svg
Is it possible to achieve this with Ambassadord?
What are your thoughts on the subject?
I think I must be missing something here. I thought ambassadord would help communicate across multiple (coreos) hosts.
So far I have the following coreos services setup:
https://gist.github.com/jeffutter/422ce28865aff09464a1
With this setup I can do the following on either host and it works:
core@core-02 ~ $ docker run --rm --link backends:backends -e "BACKEND_6379=etcd://172.17.8.101:4001/services/redis/redis" -ti redis:2.8 bash -c 'redis-cli -h backends ping'
PONG
However, the following also works, removing the utility of ambassador all-together:
core@core-02 ~ $ docker run --rm -ti redis:2.8 redis-cli -h $(etcdctl get services/redis/redis | cut -d ':' -f 1) ping
PONG
I don't see the purpose of ambassador in this setup. I assumed ambassador might work in a fashion where my docker containers could talk to ambassador via the internal docker network. Containers on the the same host would hit the local ambassador and be routed to other containers on the same host as needed. Containers on remote hosts would hit their local ambassador, which would forward to the remote ambassador (port 10000) and then forward on to the internal docker networks. Am I missing something as to how this is supposed to work?
Been using docker 1.8 (1.8.1 but I believe this is a 1.8 change) and I had to change connectable to detect NetworkMode=("bridge" || "default") because of the new networking layer. Are you interested in these changes? Since docker 1.8 is now a stable release, I'd reckon supporting it is a priority.
Containers
When I try and jump into a simple ubuntu container so I can verify the "connection" is happening, connectable always throws the error that it is unable to join a network because the container is not running.
docker run -it --rm -l connect[6379]=redis.service.consul ubuntu /bin/bash
I assume I am missing something, but cannot seem to figure it out yet.
Hi.
First, thanks for the great work.
I failed using ambassadord with Consul and registrator. I guess it is not an issue, and I did something wrong, but I cannot find in the documentation why it fails. So this is a call for help, or a full step by step tutorial ;)
Here are the steps, on a single host
Start Consul
sudo docker run -d -h consul-master -v /mnt:/data -p 8300:8300 -p 8301:8301 -p 8301:8301/udp -p 8302:8302 -p 8302:8302/udp -p 8400:8400 -p 8500:8500 -p 8600:53/udp progrium/consul -server -bootstrap
Start registrator
sudo docker run -d -v /var/run/docker.sock:/tmp/docker.sock -h $HOSTNAME progrium/registrator consul://54.77.108.120:8500
Start Redis
sudo docker run -d -p 6379 --name redis redis
Registrator makes the new redis service available in Consul.
curl http://127.0.0.1:8500/v1/catalog/service/redis
[{"Node":"consul-master","Address":"172.17.1.14","ServiceID":"ip-172-30-0-58:redis:6379","ServiceName":"redis","ServiceTags":null,"ServicePort":49157}]
Start ambassadord in point to point mode using SRV
sudo docker run --name amb -d progrium/ambassadord redis.service.consul
Start a redis-client, using docker links
sudo docker run --link amb:db redis redis-cli -h db -p 10000 ping
Error: Connection reset by peer
Checking redis is working
sudo docker run --link redis:db redis redis-cli -h db ping
PONG
Gathering the logs from ambassadord
sudo docker logs amb
2014/09/18 12:11:23 Ambassadord listening on 10000 using redis.service.consul ...
2014/09/18 12:12:29 dns: lookup redis.service.consul: no such host
2014/09/18 12:12:29 No backends! Closing the connection
Same goes for omnimod.
Thansk for your help
Started ambassadord in omni mode:
/usr/bin/docker run --name backends -v /var/run/docker.sock:/var/run/docker.sock progrium/ambassadord --omnimode
/usr/bin/docker run --rm=true --privileged --net container:backends progrium/ambassadord --setup-iptables
Then I tried to connect to something:
$ docker run --rm -it --link backends:backends -e BACKEND_4161="etcd://127.0.0.1:4001/services/foo-4161" busybox
/ # nc $BACKENDS_PORT_10000_TCP_ADDR 4161
Then I get this error from ambassadord, and the ambassador container exits:
2014/07/25 06:15:51 unrecognized config store backend: etcd
This is using the current docker image pushed to the registry. Maybe it's out of date? I'll try building it tomorrow.
Since you already have the docker socket, you can do this. See moby/moby#7228 (comment).
This would make using ambassadord much easier. If you don't have the cycles to do this, I can take a look at it this weekend, perhaps this is a good time for me to finally learn go.
I wonder if this is the right setup for consul connectable service.
Host/VM 1 : Consul server
Host/VM 2 : Consul Agent (client mode) , registrator and connectable and All micro-services
So when I started my micro-services containers, I could see them in consul server.
Question :
If I have two micro-services.... how do I load balance them ?
What API (rest API) should I call to get the services in a round robin fashion ?
How does connetable helps ?
Since connectable is sitting on each host ...does it do roundrobin of micro-services distributed across the hosts ?
Praveen
Does ambassadord expose any critical information to the containers linking to it?
After upgrading to docker 1.8.2, when the container is first created its NetworkMode is empty. Only after starting does it get its value. I'm assuming both this and #23 are due to the network system changes, not sure if this is a docker bug, or if we should check for "" NetworkMode. In my fork, i changed connectable to detect "start" instead of "create" which fixes this bug, and has the advantage of working through container restarts.
Hi,
I am trying to use this sidecar container in an architecture with Amazon EC2 Container Service (ECS). When I try to configure an ECS task definition with the label format connect[8081]=stock-price.service.consul
it gives the following error message:
Docker label key connect[8081] contains invalid characters, does not match pattern ^[a-z0-9-.]+$
According to the docker spec the ECS service is correct in checking that the labels contain only lower-cased alphanumeric characters, dots and dashes (see: https://docs.docker.com/engine/userguide/labels-custom-metadata/). It seems however that docker engine is not strict in enforcing this.
Is there any chance this can be changed to support valid Docker tag as per the spec?
When I use the wrong URI for etcd, get lots of these over and over and over again, and ambassadord can't connect to anything else.
Jul 25 21:24:47 core-01 bash[572]: 2014/07/25 21:24:46 configstore:
Jul 25 21:24:47 core-01 bash[572]: 2014/07/25 21:24:46 etcd: 501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]
When it's correct, this doesn't happen.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.