IP Address caching and docker re-using IP addresses may cause inconsistent credentials being returned about metadataproxy HOT 5 CLOSED

Taking a deeper look, I believe the issue I'm seeing is right around her: https://github.com/lyft/metadataproxy/blob/master/metadataproxy/roles.py#L92

if ip in CONTAINER_MAPPING:
        log.info('Container id for IP {0} in cache'.format(ip))
        try:
            with PrintingBlockTimer('Container inspect'):
                container = client.inspect_container(CONTAINER_MAPPING[ip])
            return container
        except docker.errors.NotFound:
            msg = 'Container id {0} no longer mapped to {1}'
            log.error(msg.format(CONTAINER_MAPPING[ip], ip))
            del CONTAINER_MAPPING[ip]

On this line, when it does the inspect_container, the container technically still exists (it's just stopped), so the except case won't get invoked if I'm correct. Probably just need to add a couple of lines to sanity check that the IP address is still assigned to that container upon docker inspect, and that the container is still running. If neither case is true, just let it continue with the remainder of the workflow and update the caches accordingly.

from metadataproxy.

p.s. - looking forward to another pair of eyes on this. I am still pretty new to this one and could be completely wrong

from metadataproxy.

I was just looking at the same lines of code and thinking "I wonder if the container still exists...". It looks like we should be checking the container's state, rather than just relying on it existing or not existing. Good find. I'll try to get a fix in for this soon!

from metadataproxy.

If you'd like to fix this before I get to it, send in a PR. I'd be happy to merge. We do have a CLA that needs to be accepted first though: https://oss.lyft.com/cla

from metadataproxy.

If nobody beats me to it, when I get back from my long weekend, I'll take a stab at it early next week. Thank you!

from metadataproxy.