Deployment on Heroku about cuttle HOT 13 CLOSED

Hi @andresdouglas, afaik, Heroku reads the Host: HTTP header to route the HTTP request. Since this is a proxy application, the value of the Host: header would always be the domain name of upstream. I had a previous experiment at https://github.com/mrkschan/tornado-http-proxy.

BTW, I'm blocked on this project by not able to make a decision on how to support more rate limiting options. I cannot conclude that if I should create (1) service-oriented rate limiters (e.g. Shopify API rate limiter, Google API rate limiter, etc.) or (2) generic rate limiters.

The obvious downside of having service-oriented limiters is duplicated logic. Though, having generic limiters may not be able to support that many APIs in the wild.

Do you have any idea here ^^?

from cuttle.

Hey @mrkschan I guess it depends how you would implement the generic rate limiters. Would they be easily configured for each service?

So just to clarify, you're saying that I can't deploy this on Heroku, correct? What other service would you suggest for it?

from cuttle.

I'm not sure which PaaS support hosting HTTP proxy. I guess the easier and
cheapest way is to use a OpenVZ provider instead. Since this project just
need a simple binary file to run.

On Sat, May 14, 2016, 07:48 Andres Douglas [email protected] wrote:

Hey @mrkschan https://github.com/mrkschan I guess it depends how you
would implement the generic rate limiters. Would they be easily configured
for each service?

So just to clarify, you're saying that I can't deploy this on Heroku,
correct? What other service would you suggest for it?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#15 (comment)

from cuttle.

OK, I've deployed to a box on Digital Ocean, so there should be no HTTP proxy issues. I am trying to figure out the SSL certificates though. Getting the following error from django

And on cuttle's side - ] Listening on :3128
2016/05/18 19:22:32 [001] WARN: Cannot handshake client domain.myshopify.com:443 remote error: unknown certificate authority

I'm guessing the question is now if there is a way to run django and specify the cacert when running it in a similar way to the last line of https://github.com/mrkschan/cuttle#suggested-setup. Do you have any ideas? If pyactiveresource were using the requests library it could be done, but it just doesn't seem to be the case

from cuttle.

Kind of hitting a wall with this. Stack Overflow isn't helping either. Any ideas?

from cuttle.

@andresdouglas: you may find https://github.com/mrkschan/cuttle/blob/master/loadtest/shopify/apicall.py as an example to specify the cacert in Python.

from cuttle.

thanks for that pointer @mrkschan !
I created the certificate on the server with

openssl req -x509 -nodes -sha1 -newkey rsa:2048 -out cacert.pem -outform PEM -days 1825

Updated the cuttle.yml to be:

$ cat cuttle.yml

addr: :3128
cakey: './privkey.pem'
zones:
  - host: "*.myshopify.com"
    shared: false
    control: rps
    rate: 2

I run my django server:
https_proxy='cuttle.mydomain.com:3128' python manage.py runserver
but get the following error:
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

On the cuttle server's end I get:
2016/06/07 19:15:56 [003] WARN: Cannot handshake client bakdrop-staging.myshopify.com:443 remote error: unknown certificate authority

Am I missing something?

from cuttle.

@andresdouglas did u specify the cacert in Python? That requires editing Python logic.

https://github.com/mrkschan/cuttle/blob/master/loadtest/shopify/apicall.py#L18-L20

And, what is your python version?

https://github.com/mrkschan/cuttle/blob/master/loadtest/shopify/apicall.py#L25-L30

from cuttle.

yep, made those changes. I've tried supplying the cacert as a file (cacert.pem is in same dir as manage.py, so simple as "cacert.pem" and also as full path to the file) and as the cadata (turned it into a one-line string replacing line breaks with \\n), but getting the same result either way. Interestingly if I give it a wrong or inexistent file name, it throws back the same error, so that's not very helpful.

Using Python 2.7.10.

from cuttle.

@andresdouglas can u just run https://github.com/mrkschan/cuttle/blob/master/loadtest/shopify/apicall.py and see what you get?

from cuttle.

If I don't run it with HTTPS_PROXY='cuttle.mydomain.com:3128' it prints out the dict of the store info. When I run it pointing at the proxy it only prints out the <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

Would that have anything to do with the error cuttle is throwing: WARN: Cannot handshake client bakdrop-staging.myshopify.com:443 remote error: unknown certificate authority?

from cuttle.

@andresdouglas I cannot reproduce your problem.

You have to start cuttle with the cuttle.yml in https://github.com/mrkschan/cuttle/blob/master/loadtest/shopify/. And, then create the env.sh with your store info. Then, lastly run https_proxy=localhost:3128 bash -c 'source env.sh && env/bin/python apicall.py' which the Python script would use cacert.pem in that folder as well.

If you provide your custom SSL cert, make sure you updated cuttle.yml to use that cert and its key.

from cuttle.

ok! Seems like I was missing specifying the cacert.pem in cuttle.yml ... :) sorry about that and many thanks for your help!

from cuttle.