Requests lock up when multiple are done in concurrent threads about google-api-python-client HOT 8 CLOSED

yes, apiclient is tied to httplib2, which isn't threadsafe. if you make separate httplib2.Http instances per thread, you'll hit less trouble.

the right answer here is to switch to something like requests, but that's quite a large change.

from google-api-python-client.

Can you point me in the right direction on doing that? Thanks!

from google-api-python-client.

on the "separate instances per thread", do you control thread creation in flask or does it handle it for you?

on the "switch to requests" front, we need to start with oauth2client. this is an extremely involved set of changes.

from google-api-python-client.

I dug around with switching to requests, but ultimately decided it was just going to be too much for me to handle. Would be great though.

Flask controls thread creation. Any way to just pass in the new httplib2.Http object on execute()?

from google-api-python-client.

yeah, the requests switch is a pretty serious set of changes. i've been hoping to start on it in oauth2client for almost a year now, and it's always "ooh ... that looks serious."

execute() actually already does take an optional http argument; probably the one hitch is that you need to make sure it's authorized.

from google-api-python-client.

Yeah, just tried that it was the problem I ran into. Any way to transfer the authorization from the default http argument to a newly initialized one?

from google-api-python-client.

it's a little convoluted, but you could do

client.something().something().execute(http=client.http.credentials.authorize(http=httplib2.Http()))

however, you might need to create the Http instance as a thread local? in any event, the old http has a handle on the credentials used to authorize it, which you can use here.

here we're moving the problem around a little -- multiple threads trying to refresh the credentials at once may hit trouble -- but it's less problematic than multiple threads sharing the same http connection pool (which is always bad juju).

from google-api-python-client.

That worked, thanks!

from google-api-python-client.