Daemonizing after importing paramiko breaks the random number generator about paramiko HOT 8 CLOSED

I also had a similar issue with a Django site running under gunicorn. Moving the import into my view method solved the issue.

from paramiko.

tl;dr: I'd argue the problem here is the blanket closing of all file descriptors.

I can't think of a great way to guard against that within Paramiko itself (offhand) and the "best" way to try reopening the file descriptor when needed, would be in Crypto, which I personally have no control over.

A quick Google shows that people recognize the need to exclude files from the 'close all open file descriptors' step (e.g. one of the points in this SO answer).

In fact, the semi-official reference implementation for Python daemonization (which IMO is probably saner to use than an ActiveState code snippet, given it has a PEP and all :)) has an explicit option for this, files_preserve, as seen in the API section.

So I'd use that PEP implementation (it's on PyPI) and figure out how to get a reference to the file handle inside the RNG. (It doesn't seem obvious when I glance at Crypto.Random; I'd bet you could get a quicker answer by debugging your "Bad file descriptor" errors.)

Alternately, keep using your existing adhoc daemonization, but make the file descriptor closure section a blacklist instead of a whitelist: identify what other open FDs your code is generating, and explicitly close just those FDs.

I'll close the ticket as I don't think there's any good action to take on our end, but please keep me/us updated with what you find, and good luck!

from paramiko.

The problem is that there's no good way to get the FD from the RNG object. The attribute name is '__file' so it gets obfuscated. The the real problem is global state is being introduced from an import. What you should do is have each Transport object have its own RNG or have them share one through some kind of cache mechanism that doesn't create the RNG until it's needed.

from paramiko.

@xraj I was worried it might be something like that. I don't have time right now to chase this down, and I'm slightly wary that modifying how Paramiko uses the RNGs could introduce other bugs -- but if you or somebody else submits a patch exploring that change I would definitely reopen this issue & take a look at it.

from paramiko.

I'll look into making a patch; for now, I have a hacky work around.

import os
from resource import getrlimit, RLIMIT_NOFILE

def files_preserve_by_path(*paths):
    wanted=[]
    for path in paths:
        fd = os.open(path, os.O_RDONLY)
        try:
            wanted.append(os.fstat(fd)[1:3])
        finally:
            os.close(fd)

    def fd_wanted(fd):
        try:
            return os.fstat(fd)[1:3] in wanted
        except OSError:
            return False

    fd_max = getrlimit(RLIMIT_NOFILE)[1]
    return [ fd for fd in xrange(fd_max) if fd_wanted(fd) ]

daemon_context.files_preserve = files_preserve_by_path('/dev/urandom')

from paramiko.

@bitprophet Is there any chance you would reconsider this ticket?
I've just run into the same daemonizing problem as the OT and I'm not very enthused to use @xraj's (although rather clever) hack.

The basic problem is that paramiko unconditionally calls Crypto.Random.new() at module level import time in common.py.
This leads to /dev/urandom being opened as soon as paramiko is imported.

As far as I can tell this "global" RNG is used in relatively few places (I found 7 files that refer it).
Would you accept a PR that changes those places to use their own dedicated RNG instance instead?

from paramiko.

@ulope That sounds reasonable to me offhand, yea. I'm still working through my backlog, if that PR got opened already please link it here, and if not, please feel free to open one. Thanks!

from paramiko.

This is also the case when you work with GCP in libcloud from inside the daemon: https://github.com/apache/libcloud/blob/8bedf2472401186ca1038719a0cdd66155f833f5/libcloud/common/google.py#L96

from paramiko.