Comments (15)
medmunds
commented on June 4, 2024
1
If your Django app doesn't know its own public hostname (because Django is behind a proxy that doesn't set X-Forwarded-Host, e.g.), you'll need to set MANDRILL_WEBHOOK_URL to the exact webhook url you entered into Mandrill's dashboard.
Without that setting, Mandrill is computing a signature for something like https://yoursite.example.com/anymail/mandrill/tracking/, but Anymail thinks it was called at something like http://127.0.0.1:8000/anymail/mandrill/tracking/ -- so the signatures don't match.
(I'm making a note to update that error message to include the url Anymail thinks it was called at.)
If that's not the issue, it would be helpful to know things like Anymail and Django versions, whether the webhook has ever worked (and if so, what changed since then).
You could also scan up the stack trace in Sentry to find where anymail.webhooks.mandrill...validate_request is raising the error -- Sentry may have captured more information there that would help your investigation. In particular, signed_data will start with the url Anymail thinks it was called at (followed by your webhook key and the request params that were signed).
from django-anymail.
patroqueeet
commented on June 4, 2024
dear, the x-forwarded-host is properly set to the domain of the webhook. but I realized that in mandrill, the webhook url registered contains the authentication credentials (as of your documentation). if that is used for signing, then it won't work. what do you think?
also signed_data is not reported to sentry. only mandrill_events, request headers and env variables are recorded. there stack trace is also missing. this happens usually if the exception is catched somewhere else and a logger reports the failure. in the sentry logger name is django.security.AnymailWebhookValidationFailure. what is the best way to have an insigt into signed_data?
and no, I just switched to anymail and it was never successfull.
from django-anymail.
patroqueeet
commented on June 4, 2024
Update: I disabled WEBHOOK_AUTHORIZATION setting, removed the httpauth from the mandrill registered url and tried again. but that failed again with a 400.
from django-anymail.
patroqueeet
commented on June 4, 2024
another Update: I managed to print signed_data on the instance:
u'https://SUBDOMAIN.DOMAIN.com/webhooks/mail/mandrill/tracking/mandrill_events[{"event":"send","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"opens":[],"clicks":[],"state":"sent","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa","_version":"exampleaaaaaaaaaaaaaaa"},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa","ts":1483429533},{"event":"deferral","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"opens":[],"clicks":[],"state":"deferred","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa1","_version":"exampleaaaaaaaaaaaaaaa","smtp_events":[{"destination_ip":"127.0.0.1","diag":"451 4.3.5 Temporarily unavailable, try again later.","source_ip":"127.0.0.1","ts":1365111111,"type":"deferred","size":0}]},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa1","ts":1483429533},{"event":"hard_bounce","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"state":"bounced","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa2","_version":"exampleaaaaaaaaaaaaaaa","bounce_description":"bad_mailbox","bgtools_code":10,"diag":"smtp;550 5.1.1 The email account that you tried to reach does not exist. Please try double-checking the recipient\'s email address for typos or unnecessary spaces."},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa2","ts":1483429533},{"event":"soft_bounce","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"state":"soft-bounced","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa3","_version":"exampleaaaaaaaaaaaaaaa","bounce_description":"mailbox_full","bgtools_code":22,"diag":"smtp;552 5.2.2 Over Quota"},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa3","ts":1483429533},{"event":"open","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"opens":[{"ts":1365111111}],"clicks":[{"ts":1365111111,"url":"http:\\/\\/mandrill.com"}],"state":"sent","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa4","_version":"exampleaaaaaaaaaaaaaaa"},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa4","ip":"127.0.0.1","location":{"country_short":"US","country":"United States","region":"Oklahoma","city":"Oklahoma City","latitude":35.4675598145,"longitude":-97.5164337158,"postal_code":"73101","timezone":"-05:00"},"user_agent":"Mozilla\\/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.8) Gecko\\/20100317 Postbox\\/1.1.3","user_agent_parsed":{"type":"Email Client","ua_family":"Postbox","ua_name":"Postbox 1.1.3","ua_version":"1.1.3","ua_url":"http:\\/\\/www.postbox-inc.com\\/","ua_company":"Postbox, Inc.","ua_company_url":"http:\\/\\/www.postbox-inc.com\\/","ua_icon":"http:\\/\\/cdn.mandrill.com\\/img\\/email-client-icons\\/postbox.png","os_family":"OS X","os_name":"OS X 10.6 Snow Leopard","os_url":"http:\\/\\/www.apple.com\\/osx\\/","os_company":"Apple Computer, Inc.","os_company_url":"http:\\/\\/www.apple.com\\/","os_icon":"http:\\/\\/cdn.mandrill.com\\/img\\/email-client-icons\\/macosx.png","mobile":false},"ts":1483429533},{"event":"click","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"opens":[{"ts":1365111111}],"clicks":[{"ts":1365111111,"url":"http:\\/\\/mandrill.com"}],"state":"sent","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa5","_version":"exampleaaaaaaaaaaaaaaa"},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa5","ip":"127.0.0.1","location":{"country_short":"US","country":"United States","region":"Oklahoma","city":"Oklahoma City","latitude":35.4675598145,"longitude":-97.5164337158,"postal_code":"73101","timezone":"-05:00"},"user_agent":"Mozilla\\/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.8) Gecko\\/20100317 Postbox\\/1.1.3","user_agent_parsed":{"type":"Email Client","ua_family":"Postbox","ua_name":"Postbox 1.1.3","ua_version":"1.1.3","ua_url":"http:\\/\\/www.postbox-inc.com\\/","ua_company":"Postbox, Inc.","ua_company_url":"http:\\/\\/www.postbox-inc.com\\/","ua_icon":"http:\\/\\/cdn.mandrill.com\\/img\\/email-client-icons\\/postbox.png","os_family":"OS X","os_name":"OS X 10.6 Snow Leopard","os_url":"http:\\/\\/www.apple.com\\/osx\\/","os_company":"Apple Computer, Inc.","os_company_url":"http:\\/\\/www.apple.com\\/","os_icon":"http:\\/\\/cdn.mandrill.com\\/img\\/email-client-icons\\/macosx.png","mobile":false},"url":"http:\\/\\/mandrill.com","ts":1483429533},{"event":"spam","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"opens":[{"ts":1365111111}],"clicks":[{"ts":1365111111,"url":"http:\\/\\/mandrill.com"}],"state":"sent","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa6","_version":"exampleaaaaaaaaaaaaaaa"},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa6","ts":1483429533},{"event":"unsub","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"opens":[{"ts":1365111111}],"clicks":[{"ts":1365111111,"url":"http:\\/\\/mandrill.com"}],"state":"sent","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa7","_version":"exampleaaaaaaaaaaaaaaa"},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa7","ts":1483429533},{"event":"reject","msg":{"ts":1365109999,"subject":"This an example webhook message","email":"[email protected]","sender":"[email protected]","tags":["webhook-example"],"opens":[],"clicks":[],"state":"rejected","metadata":{"user_id":111},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa8","_version":"exampleaaaaaaaaaaaaaaa"},"_id":"exampleaaaaaaaaaaaaaaaaaaaaaaaaa8","ts":1483429533},{"type":"blacklist","action":"add","reject":{"reason":"hard-bounce","detail":"Example detail","last_event_at":"2014-02-01 12:43:56","email":"[email protected]","created_at":"2014-01-15 11:32:19","expires_at":"2020-04-02 12:09:18","expired":false,"subaccount":"example_subaccount","sender":"[email protected]"},"ts":1483429533},{"type":"blacklist","action":"change","reject":{"reason":"hard-bounce","detail":"Example detail","last_event_at":"2014-02-01 12:43:56","email":"[email protected]","created_at":"2014-01-15 11:32:19","expires_at":"2020-04-02 12:09:18","expired":false,"subaccount":"example_subaccount","sender":"[email protected]"},"ts":1483429533},{"type":"blacklist","action":"remove","reject":{"reason":"hard-bounce","detail":"Example detail","last_event_at":"2014-02-01 12:43:56","email":"[email protected]","created_at":"2014-01-15 11:32:19","expires_at":"2020-04-02 12:09:18","expired":false,"subaccount":"example_subaccount","sender":"[email protected]"},"ts":1483429533},{"type":"whitelist","action":"add","entry":{"email":"[email protected]","detail":"example details","created_at":"2014-01-15 12:03:19"},"ts":1483429533},{"type":"whitelist","action":"remove","entry":{"email":"[email protected]","detail":"example details","created_at":"2014-01-15 12:03:19"},"ts":1483429533}]'
does this look right?
I also printed the expected_signature and the signature. they were both different.
from django-anymail.
patroqueeet
commented on June 4, 2024
while investigating I noticed: in sentry the mandrill_events contains http:\/\/www.apple.com\/ but inside the signed_data it is: http:\\/\\/www.apple.com\\/
from django-anymail.
medmunds
commented on June 4, 2024
Ah -- I think you're correct about basic auth affecting how Mandrill calculates the signature. (But I can't really tell from Mandrill's docs, and I don't have access to Mandrill for testing.)
But yes, let's try to get it working without basic auth first. Where you've added print debugging in MandrillSignatureMixin.validate_request, let's also print some other things and verify they're correct:
print self.basic_authshould be[](because you disabledWEBHOOK_AUTHORIZATION)print self.webhook_keyshould be the exact current webhook key from the Mandrill webhook dashboard. (Mandrill may have re-generated this key when you changed the webhook url to remove basic auth, so it wouldn't hurt to check again.)print self.webhook_urlshould be'https://SUBDOMAIN.DOMAIN.com/webhooks/mail/mandrill/tracking/'-- and that should be the exact url in Mandrill's webhooks dashboard, too -- please double-check 'https:' vs 'http:', and that both have a trailing '/'.print request.build_absolute_uri()ideally would be the same asself.webhook_url. (But if not, the difference would tell us why you're needing to setMANDRILL_WEBHOOK_URL.)
If none of those shows a problem, I'm afraid I'm a bit mystified.
from django-anymail.
patroqueeet
commented on June 4, 2024
hey, thanks for the guidance. will instantly tomorrow morning. seen my comment about the escaping?
from django-anymail.
medmunds
commented on June 4, 2024
I think the escaping difference is just an artifact of printing to the console.
from django-anymail.
patroqueeet
commented on June 4, 2024
hey, self.webhook_url is None, but according to signed_data = self.webhook_url or request.build_absolute_uri() this should be fine.
with printing I get
auth ['AUTH:AUTH2']
hook key 'KEY'
hook url None
uri `https://SUBDOMAIN.DOMAIN.com/webhooks/mail/mandrill/tracking/`
I compared the url 20 times with mandrill and it looks fine. even with the trailing slash.
from django-anymail.
patroqueeet
commented on June 4, 2024
alright I finally got it: when using the authentication mechanism the url for the signed_data is retrieved like this: https://github.com/anymail/django-anymail/blob/master/anymail/webhooks/mandrill.py#L49 which means it either is using the automatically created uri or the setting. I had the setting empty (as you documented it optional). but the created uri does not contain the authentication data. hence, exaclty as documented ( https://mandrill.zendesk.com/hc/en-us/articles/205583257-How-to-Authenticate-Webhook-Requests#generate-a-signature ) the uri used MUST contain the authentication data (even if mandrill is hiding it inside it's own UI).
to make it short adding the same URI as entered into mandrill form (including the authentication data) as a setting for MANDRILL_WEBHOOK_URL did the trick for me.
maybe it would make sense to have this documented, that if you are using the authentication, the setting would be mandatory.
from django-anymail.
medmunds
commented on June 4, 2024
Ah, great. Thanks for tracking this down, and glad you were able to get it working.
I'm going to treat this as a bug in the Anymail's Mandrill webhook verification. If Anymail knows you're using basic auth (and it does!), it should factor that into the signature calculation.
For anyone else encountering this, until there's a fix in place, the workaround is to set MANDRILL_WEBHOOK_URL to include your WEBHOOK_AUTHORIZATION secret:
ANYMAIL = {
"MANDRILL_API_KEY": "<your API key>",
"WEBHOOK_AUTHORIZATION": "random:random", # use same random:random below
"MANDRILL_WEBHOOK_URL": "https://random:[email protected]/anymail/mandrill/tracking/",
# ...
}from django-anymail.
medmunds
commented on June 4, 2024
@patroqueeet you mentioned "authentication data (even if mandrill is hiding it inside it's own UI)"... does that mean the Mandrill webhooks settings is extracting the basic auth from the url and showing it in some other UI fields? (Does it have a separate place for webhook username/password?)
I'd like to clarify Anymail's docs if that's how Mandrill handles webhook basic auth. If you could let me know the exact labels for things in the Mandrill webhook page -- or post a screenshot (empty webhooks form or sensitive data removed) -- that would be really helpful.
Thanks.
from django-anymail.
patroqueeet
commented on June 4, 2024
hey, current mandrill form to add a webhook: https://www.dropbox.com/s/ec96qromh6rne76/Screenshot%202017-01-24%2007.50.15.png?dl=0 and this is the display. looks like they altered it. when we talked here, the auth data was not reflected by the status display url.
btw. you are the first ever open source maintainer which reflected a ticket and came back to me with the goal to make his great library even better...
from django-anymail.
medmunds
commented on June 4, 2024
Thanks, looks like Mandrill has renamed a couple of things since the last time I saw that screen. I'll update the docs.
I really appreciate all the effort you put into researching this issue. It's been tricky maintaining support for Mandrill without access to their API, and it would be absolutely impossible without the help of users like you.
from django-anymail.
patroqueeet
commented on June 4, 2024
I'm working with them every single day. ping me whenever you need any insights...
from django-anymail.
Related Issues (20)
- Support Brevo inbound
- "Package would be ignored" warnings from setuptools when building package HOT 2
- Sendgrid multiple reply to HOT 1
- Can't lock with poetry HOT 2
- [Inbound] Apple Mail inline attachments not taken into account HOT 5
- Add support to Resend.com HOT 5
- Handlebars variables not substitued in mandrill templates HOT 1
- mailgun api key HOT 2
- Merging/Combine ESP Options get lost/overwritten HOT 2
- parse_address_list: Inability to Parse Email Addresses with Special Characters in Display Name Without Double-Quote Encapsulation HOT 1
- Mailjet Open tracking? HOT 4
- Inaccuracy in documentation about Brevo's behaviour HOT 2
- Feature request: Mailtrap HOT 4
- Resend: confusing ReadTimeout error for incorrect API key
- Add support to Unisender Go HOT 2
- Support Brevo batch sending with merge_data HOT 2
- Add batch sending support for Resend HOT 1
- Mailgun webhooks: AttributeError 'NoneType' object has no attribute 'get' HOT 1
- boto3 sesv2 client does not have method 'send_email'. Check AmazonSESV2SendEmailPayload HOT 4
- Bad UTF-8 "To" header encoding? HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from django-anymail.