Example in README doesn't work. about docker-python-sandbox HOT 17 CLOSED

Thank you @murtyjones for the informative fork and @christophetd for this excellent starting point for using docker with node!

from docker-python-sandbox.

Thanks! I added a warning for Mac users in the readme.

from docker-python-sandbox.

No worries - I appreciate the attempt to help.

I will update if I find a solution or a reasonable workaround for Mac users.

from docker-python-sandbox.

From my initial messing about with the docker image, it seems like a port binding may be needed as part of the launch config. Otherwise the container isn't reachable.

EDIT - adding the following to the HostConfig portion of the container options binds the ports and exposes localhost:8080 to the server to make requests:

"PortBindings": {
  "3000/tcp": [{
    "HostPort": "8080"
  }]
}

only issue is that i can't see how one would spin up multiple containers in the pool as they would all need to be bound to different ports. Is there something I'm missing about how to get access to the container's node server from the host?

from docker-python-sandbox.

@christophetd do you know of any instances of successful usage of this library on a Mac? I'm thinking that my issue relates to the fact that I'm on a Mac and this is geared towards Linux. Don't know why specifically yet.

from docker-python-sandbox.

Hi @murtyjones,

I probably won't be shipping new features for this library, but I'm potentially ready to fix bugs. I don't know anyone who tried or succeeded to run it on a Mac, unfortunately, and I can't provide support since I don't own one. However I can confirm that the example code runs as expected on a basic Ubuntu 16.04 box running Docker CE 17.06 and node v6.11.3.

Regarding your question on port bindings, an explicit port binding is only necessary if you want your containers' ports to be mapped to a host port. Even without a port binding, you can access any port of a container from the host using the container's private IP (which is the way I implemented it in this library). Here are a few commands demonstrating this:

# 172.17.0.0/16 is Docker's private IP range and gets routed to Docker
$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
...
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
...

$ docker run -d --name nginx --rm nginx
$ docker inspect --format '{{ .NetworkSettings.IPAddress }}' nginx
172.17.0.2
$ curl 172.17.0.2
...
<h1>Welcome to nginx!</h1>
...

Hope that helps! Let me know if you have any other question.

from docker-python-sandbox.

Some additional questions to help debug your issue:

• Do you have any error message coming from the library?
• Are the containers correctly created? (if you run docker ps while running the example code)
• Could you run the following commands and let me know how it works out?

$ docker run -d --rm --name test-container christophetd/docker-sandbox
$ ip=$(docker inspect --format '{{ .NetworkSettings.IPAddress }}' test-container)
$ curl -X POST $ip:3000 -H "Content-Type: application/json" -d '{"code": "print 21*2", "timeoutMs": "1000"}'
{"stdout":"42\n","stderr":"","combined":"42\n","isError":false}
$ docker rm -f test-container # cleanup

As you can see, this last snippet reproduces what the library does internally.

from docker-python-sandbox.

Hey - thanks for the response.

Looks to me like the containers are spinning up correctly (I'm seeing five of them which makes sense given the default).

The commands you provided yield this:

{"stdout":"42\n","stderr":"","combined":"42\n","isError":false}

No error messages coming from the library that i can see.

I'm wondering if my containerLaunchOptions should be different on a Mac, but not sure how that would look. I also tried switching from the overlay2 storage-driver to aufs, but no luck there.

from docker-python-sandbox.

Looks like the containers are indeed running correctly and accepting compilation commands, so I agree it might be containerLaunchOptions related. Can you try the following and paste the contents of debug.log here?

# Shell 1
while true; do
    for container_id in $(docker ps -aq); do
        docker inspect $container_id >> debug.log
    done
    sleep 2;
    echo -e "\n\n"
done


# Shell 2
node example.js

(ctrl+c the first shell once the example code has finished running)

from docker-python-sandbox.

Thanks for the suggestion - the debug.log is attached.
debug.log

from docker-python-sandbox.

Weird. From a quick glance, I can see that the containers are correctly assigned an IP address:

"Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "3ea8d2a775a166c7b0cc208f7f6076a6b8c034cc456adaea0fe2d7bda032dada",
                    "EndpointID": "d3cfd848aa82055e3a855cda5f1afe828104de13196ebf0a11918c576b98b1dd",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.5",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:05",
                    "DriverOpts": null
                }
            }

Can you paste the output of netstat -nr? This should output your computer's routing table, I just want to make sure that the IP addresses of the containers (172.17.0.5/24 or /16) are correctly routed via your docker interface.

from docker-python-sandbox.

Sure - here's the output:

Routing tables

Internet:
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            192.168.0.1        UGSc           17        0     en0
127                127.0.0.1          UCS             0        0     lo0
127.0.0.1          127.0.0.1          UH              4   675756     lo0
169.254            link#5             UCS             0        0     en0
192.168.0          link#5             UCS             3        0     en0
192.168.0.1/32     link#5             UCS             1        0     en0
192.168.0.1        b0:7f:b9:7d:ce:b2  UHLWIir        18       33     en0    402
192.168.0.12       d8:bb:2c:23:6c:a0  UHLWIi          1       16     en0    339
192.168.0.13       0:db:70:8f:4b:e2   UHLWI           0        0     en0    405
192.168.0.14       28:b2:bd:b2:b2:9a  UHLWI           0        0     en0    155
192.168.0.17/32    link#5             UCS             0        0     en0
224.0.0/4          link#5             UmCS            3        0     en0
224.0.0.251        1:0:5e:0:0:fb      UHmLWI          0        0     en0
230.230.230.230    1:0:5e:66:e6:e6    UHmLWI          0       28     en0
239.255.255.250    1:0:5e:7f:ff:fa    UHmLWI          0      153     en0
255.255.255.255/32 link#5             UCS             0        0     en0

Internet6:
Destination                             Gateway                         Flags         Netif Expire
default                                 fe80::b27f:b9ff:fe7d:ceb2%en0   UGc             en0
default                                 fe80::%utun0                    UGcI          utun0
default                                 fe80::%utun1                    UGcI          utun1
::1                                     ::1                             UHL             lo0
2605:6000:e98e:f800::/64                link#5                          UC              en0
2605:6000:e98e:f800:1050:5190:5972:fb1c d8:bb:2c:23:6c:a0               UHLWI           en0
2605:6000:e98e:f800:145d:b3af:ea9f:586d 8c:85:90:1a:94:5c               UHL             lo0
2605:6000:e98e:f800:1d2f:587e:8899:28b2 8c:85:90:1a:94:5c               UHL             lo0
fe80::%lo0/64                           fe80::1%lo0                     UcI             lo0
fe80::1%lo0                             link#1                          UHLI            lo0
fe80::%en5/64                           link#4                          UCI             en5
fe80::aede:48ff:fe00:1122%en5           ac:de:48:0:11:22                UHLI            lo0
fe80::aede:48ff:fe33:4455%en5           ac:de:48:33:44:55               UHLWIi          en5
fe80::%en0/64                           link#5                          UCI             en0
fe80::58:4b58:4ec0:7621%en0             8c:85:90:1a:94:5c               UHLI            lo0
fe80::1036:c446:69d4:b8d2%en0           d8:bb:2c:23:6c:a0               UHLWIi          en0
fe80::b27f:b9ff:fe7d:ceb2%en0           b0:7f:b9:7d:ce:b2               UHLWIir         en0
fe80::%awdl0/64                         link#12                         UCI           awdl0
fe80::9490:dfff:fe50:2881%awdl0         96:90:df:50:28:81               UHLI            lo0
fe80::%utun0/64                         fe80::b510:cb05:ffa6:4a7d%utun0 UcI           utun0
fe80::b510:cb05:ffa6:4a7d%utun0         link#13                         UHLI            lo0
fe80::%utun1/64                         fe80::9921:25ee:5a9c:5339%utun1 UcI           utun1
fe80::9921:25ee:5a9c:5339%utun1         link#14                         UHLI            lo0
ff01::%lo0/32                           ::1                             UmCI            lo0
ff01::%en5/32                           link#4                          UmCI            en5
ff01::%en0/32                           link#5                          UmCI            en0
ff01::%awdl0/32                         link#12                         UmCI          awdl0
ff01::%utun0/32                         fe80::b510:cb05:ffa6:4a7d%utun0 UmCI          utun0
ff01::%utun1/32                         fe80::9921:25ee:5a9c:5339%utun1 UmCI          utun1
ff02::%lo0/32                           ::1                             UmCI            lo0
ff02::%en5/32                           link#4                          UmCI            en5
ff02::%en0/32                           link#5                          UmCI            en0
ff02::%awdl0/32                         link#12                         UmCI          awdl0
ff02::%utun0/32                         fe80::b510:cb05:ffa6:4a7d%utun0 UmCI          utun0
ff02::%utun1/32                         fe80::9921:25ee:5a9c:5339%utun1 UmCI          utun1

from docker-python-sandbox.

I'm not familiar with how docker runs on Mac. Do you run it directly, or via Docker machine? In either case I'm a bit confused because docker IPs 172.17.x.x don't appear in your route table.

Here's what I have on my machine:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
...
--->172.17.0.0<---      0.0.0.0         255.255.0.0     U     0      0        0 ---->docker0<----

Can you try:

docker run -d --rm --name test-container christophetd/docker-sandbox
docker inspect --format '{{ .NetworkSettings.IPAddress }}' test-container
docker rm -f test-container

To see if the IPs your containers get when you run them manually is in the same range as when they are started by the library?

from docker-python-sandbox.

172.17.0.3 is the output of that series of commands.

RE: your question, it depends on the version, but Docker for Mac (The version I use and the one that's recommended by Docker) does not use docker machine, but rather creates and manages the VM directly, using Hyperkite.

I've been wondering if using dockorde on Mac requires the docker-machine library, but the documentation is lacking so it's hard to tell. This is the one example I found in the dockerode GitHub that made me think docker-machine might be needed, but with Docker for Mac's release this year it's difficult to gauge whether/how out of date this information is.

[EDIT] Seems like this might be relevant: docker/for-mac#155

from docker-python-sandbox.

Might be related, although you are able to access the containers from their IPs as we tested earlier... Sorry, running out of ideas here.

from docker-python-sandbox.

It's a hacky solution, so I don't believe a pull request makes sense, but I've got it working locally on a Mac. Essentially Mac users who want to use this library locally need to run the docker image, bound to a port, and send all execution requests to localhost:3000. It's weird but it works for development purposes.

Instructions are detailed in my fork of the repo: https://github.com/murtyjones/docker-pypy-sandbox

from docker-python-sandbox.

This thread was very insightful, maybe you can link to it from the README where you mention the issue on Mac.

Like murtyjones, I also forked this repo and managed to make some very minor tweaks to get it to work on Mac based on what I learned here: https://github.com/jven/docker-python-sandbox.

from docker-python-sandbox.