Support for SparkPost about django-anymail HOT 17 CLOSED

@richleland I've had a chance to take a closer look at SparkPost, and it seems like a great fit for django-anymail (and a great choice as a transactional ESP).

I'd suggest using the SparkPost python bindings as the basis for an Anymail SparkPost backend, rather than writing directly to your HTTP API, for exactly the maintainability reasons you mentioned. Anymail's base backend/payload are meant to be binding- and transport-agnostic. [I originally went down the official bindings path with one of the other ESPs supported by Anymail, but backed off when it became clear their python bindings weren't actively maintained and were lacking functionality available in their raw API. It doesn't seem like that would be a problem with python-sparkpost.]

That should also help with getting Anymail users immediate access to critical fixes and future SparkPost improvements, as they can update python-sparkpost independently of Anymail releases. Also, based on experiences with Djrill/Mandrill, I'm also trying to design Anymail with some "escape hatches" (like esp_extra and esp_response) that allow users to directly access ESP API features without waiting for Anymail updates.

Lastly, I'm very open to adding collaborators to this project, and hoping it will live on outside any one organization or individual.

Let me know if I can answer any other questions. As I alluded to earlier, the docs are missing a "how to add an ESP" section right now, but most of it should be pretty straightforward. I'd be thrilled to have SparkPost support in Anymail.

from django-anymail.

I'm jumping in the conversation. As an undecided user for a new ESP , I was initially going to go for Sparkpost. But my whole application is currently based on djdrill, with the tag/metadata support. Anymail sounds a great choice for my project for an easy transition.

My application is not focused on emails, so I'm not going to spend an enormous amount of time to send AB splitted versions and to tag them on the fly. I'll probably look back at Sparkpost later.

Anyway, the concept behind anymail is really nice. I like it a lot @medmunds . I'll try it with the second ESP I had on my wishlist (which is fortunately supported).

from django-anymail.

@medmunds question - I'm starting to dig in to implementing SparkPost and I noticed that the method parse_recipient_status expects a dict with a mapping of email to status. With SparkPost we don't provide statuses for each email in our API call, rather we handle the batches async and have a response that looks like this:

{
  "results": {
    "total_rejected_recipients": 0,
    "total_accepted_recipients": 1,
    "id": "11668787484950529"
  }
}

How would you recommend handling the return value for parse_recipient_status given the response above? Also note that we don't return individual message IDs rather a transmission ID (kinda like a batch ID). Let me know what you think.

from django-anymail.

@richleland let me know if there's anything I can do to help with a SparkPost backend for Anymail. If you have a partial implementation going and want to post it as a PR, I'm happy to collaborate on finishing it up. Or if you're still in the investigation stage, just let me know -- I could get a skeleton up in a feature branch pretty quickly, and then we could figure out some of the edge cases.

from django-anymail.

I've got an initial SparkPost backend implementation up in the feature_sparkpost branch. Should be pretty complete on the backend features, but I have a few questions... will follow up after the holiday.

(Also, looks like you might be addressing the "figure out which recipients were rejected" issue 😄... I'll wait for official word before looking at rcpt_to_errors.)

[Todo list in PR #20]

from django-anymail.

We seem to have lost SparkPost from this discussion (@richleland?). I'm going to go ahead and release my SparkPost implementation as django-anymail 0.4, so we can start getting some real-world feedback.

from django-anymail.

I contacted them about spending their own engineering time on churning out this integration and they seemed very uninterested. I don't think they appreciate how many sites out there were using Djrill.

Long story short: 👍

from django-anymail.

I also just made that request to their developers.

from django-anymail.

Hi @cralston0 I'm not sure who you reached out to, but we've definitely heard this loud and clear and have been discussing adding a backend to django-anymail. Thanks for submitting the issue on our repo as well @ojwoodford.

I have a couple of concerns, both related to maintainability:

• Our Django backend just leverages the underlying Python class, which has code in it for normalizing the API requests. There might be a lot of duplicate code to maintain if we have the backend in django-anymail and python-sparkpost
• We have no control/access to this repo or when changes are made/deployed, so if there is a critical bug or issue that needs to be resolved quickly for our customers we have no way to say when it will be resolved for them

Curious - do either of you know the overall adoption of django-anymail? If we did end up adding a backend for django-anymail, would the Django backend in python-sparkpost still make sense or would pushing people towards django-anymail with a 2.0 release of python-sparkpost be the way to go?

I really like what's going on with this repo, I just need to understand more about it holistically before pulling the trigger. It really comes down to support and maintainability, not the amount of work involved (which seems pretty minimal).

from django-anymail.

Thanks for joining, @richleland! I don't have any idea of overall adoption, but the repo was forked from Djrill only a month ago, so I guess widespread adoption will take time. @medmunds might be able to shed more light on plans for support.

from django-anymail.

Hi, maintainer here. I'm on vacation right now, will jump in with more details when I return next week. @ojwoodford is right about adoption... anymail is just getting started in the post-Mandrill era. And template support is still being designed. On the backend side, it should be straightforward to work directly with a native Python API--the requests-specific stuff is all separate. Improving docs for the internals is on the todo list.

from django-anymail.

Update on what I'm doing. Currently started paying for Mandrill, and still hoping to switch to SparkPost, but may choose SendGrid instead in a few months time if need be.

from django-anymail.

@richleland great to hear!

For recipient status, you could probably do something like this:

• if total_accepted_recipients == total recipient count, use status "queued" for all recipients
• if total_rejected_recipients == total recipient count, use status "rejected" for all recipients
• in all other cases, use status "unknown" for all recipients -- because we can't known which ones were accepted and which rejected

(I think the most common case is a single recipient, so the "unknown" status should be pretty rare in practice.)

The primary point of the returned message_id is to match against later webhook status events on that message (and/or pass to status-checking APIs). So ideally, I think you'd want the message_id property that gets sent to SparkPost's webhooks. But I think transmission_id would also work, as Anymail doesn't support batching multiple messages in the same API request (transmission). (It supports sending a single message to multiple recipients at once. If I'm understanding correctly, you'd treat that as a single transmission_id.)

Worst case, we could generate a true RFC Message-ID header (assuming SparkPost allows this header from the client), and use that as the unique message_id. That's how Anymail handles lack of a unique message_id in the SendGrid backend.

from django-anymail.

@richleland a few questions:

1. I'm using SparkPost's (single) campaign for Anymail's message tags. Is there some better way to tag a message for report segmentation? (It looked like SparkPost's rcpt_tags are only attached to individual addresses in stored recipient lists, so couldn't be used for tagging messages.)

2. Is there a way to provide per-recipient substitution data with the transmissions.send call when sending to a stored recipient list? (That data has to be part of the recipients param right? And that param is ignored when recipient_list is provided.)

3. I'm seeing a 'rcpt_to_errors' field in the transmissions.send API result when there's a mixture of rejected and accepted recipients. Example:

   {'rcpt_to_errors': [{
       'description': 'invalid recipient address.email: notanemail', 
       'code': '1300', 
       'message': 'invalid data format/type'}],
    'total_rejected_recipients': 1, 'total_accepted_recipients': 6, 'id': '48314278541369152'}

   I don't see that documented anywhere. Do you think it would be reasonable to look at it (if present) to figure out which recipients were accepted or rejected?

Thanks.

from django-anymail.

Fantastic! Will it require python-sparkpost too?

from django-anymail.

Yes, it will. Install with:

pip install django-anymail[sparkpost]

from django-anymail.

Congrats for the update @medmunds . I'm switching to this implementation now.

from django-anymail.