Comments (21)
catphish
commented on May 26, 2024
1
I can give you the message source, but it doesn't include the headers, just the body
This is actually extremely useful. I have a hypothesis. We will look into it and let you know.
from postal.
adamcooke
commented on May 26, 2024
1
As @catphish said, I've pushed a fix for this and released 2.3.1 which includes that fix. Let us know how you get on with that.
Assuming it works, this issue probably sets a record for the quickest resolved issue in Postal history so congratulations. I can't guarantee any future issues will be so lucky but you never know 🤞
from postal.
adamcooke
commented on May 26, 2024
1
Yeh, I might have spoke a bit soon. It seems like the build didn't finish. It's currently building the image so it should be there soon.
from postal.
catphish
commented on May 26, 2024
1
Okay so that makes sense. We're using a Wordpress plugin on this website to generate the emails - if it's generating garbage headers then I'll file a bug with them too.
No - the original email is perfectly valid. It's Postal messing up the headers.
It's just unusual (but perfectly valid) that the Wordpress plugin generates emails with HTML only. Usually you'd send both HTML and plain text together in the same email (multipart).
from postal.
adamcooke
commented on May 26, 2024
1
@pawprintdigital The image has made its way in to the container registry now so you should be good to upgrade now.
Edit: I see you've already discovered this on your own. Good to hear it works!
from postal.
adamcooke
commented on May 26, 2024
1
Happy to create a new issue for this if I can't find an existing open issue for it.
There is an issue for that #907. It's quite an old one which escaped our attention. It'll be fixed in v3 which will be available soon.
from postal.
catphish
commented on May 26, 2024
That is interesting. Would you be able to provide a full (uncorrupted) copy of the raw email?
from postal.
pawprintdigital
commented on May 26, 2024
Hi @catphish, the "original message source" above is the uncorrupted version, with the "delivered message source" being the corrupted output that is ultimately delivered.
from postal.
catphish
commented on May 26, 2024
Hi @catphish, the "original message source" above is the uncorrupted version, with the "delivered message source" being the corrupted output that is ultimately delivered.
Sorry, I wasn't clear in my email, I meant the full raw email (with headers etc). I know you can't download it without corruption, but if you can get a copy somehow that would be very helpful.
from postal.
pawprintdigital
commented on May 26, 2024
@catphish Sure, give me 10 :)
from postal.
pawprintdigital
commented on May 26, 2024
@catphish Ok so it may not be so simple. I can capture the SMTP transcript between the sending webserver and Postal if that's what you're after. It is however formatted terribly, so I apologize in advance.
235 Granted for the-whole-shebang/outbound-1 MAIL FROM:<[email protected]> 250 OK RCPT TO:<[email protected]> 250 OK DATA 354 Go ahead X-Mailer: Postman SMTP 2.8.11 for WordPress (https://wordpress.org/plugins/post-smtp/) Content-Type: text/html; charset=UTF-8 From: The Whole Shebang <[email protected]> To: [email protected] Reply-To: [email protected] Message-Id: <[email protected]> Subject: Password Reset Request for The Whole Shebang Date: Fri, 23 Feb 2024 21:27:08 +0000 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline MIME-Version: 1.0 <!DOCTYPE html>=0A<html lang=3D"en-GB">=0A=09<head>=0A=09=09<meta http-e= quiv=3D"Content-Type" content=3D"text/html; charset=3DUTF-8">=0A=09=09<m= eta content=3D"width=3Ddevice-width, initial-scale=3D1.0" name=3D"viewpo= rt">=0A=09=09<title>The Whole Shebang</title>=0A=09<style type=3D"text/c= ss">@media screen and (max-width: 600px){#header_wrapper{padding: 27px 3= 6px !important; font-size: 24px;}#body_content_inner{font-size: 10px !im= portant;}}</style>=0A</head>=0A=09<body leftmargin=3D"0" marginwidth=3D"= 0" topmargin=3D"0" marginheight=3D"0" offset=3D"0" style=3D"background-c= olor: #f7f7f7; padding: 0; text-align: center;" bgcolor=3D"#f7f7f7">=0A= =09=09<table width=3D"100%" id=3D"outer_wrapper" style=3D"background-col= or: #f7f7f7;" bgcolor=3D"#f7f7f7">=0A=09=09=09<tr>=0A=09=09=09=09<td><!-= - Deliberately empty to support consistent sizing and layout across mult= iple email clients. --></td>=0A=09=09=09=09<td width=3D"600">=0A=09=09= =09=09=09<div id=3D"wrapper" dir=3D"ltr" style=3D"margin: 0 auto; paddin= g: 70px 0; width: 100%; max-width: 600px; -webkit-text-size-adjust: none= ;" width=3D"100%">=0A=09=09=09=09=09=09<table border=3D"0" cellpadding= =3D"0" cellspacing=3D"0" height=3D"100%" width=3D"100%">=0A=09=09=09=09= =09=09=09<tr>=0A=09=09=09=09=09=09=09=09<td align=3D"center" valign=3D"t= op">=0A=09=09=09=09=09=09=09=09=09<div id=3D"template_header_image">=0A= =09=09=09=09=09=09=09=09=09=09<p style=3D"margin-top: 0;"><img src=3D"ht= tps://thewhole-shebang.co.uk/wp-content/uploads/2023/03/wsi.jpg" alt=3D"= The Whole Shebang" style=3D"border: none; display: inline-block; font-si= ze: 14px; font-weight: bold; height: auto; outline: none; text-decoratio= n: none; text-transform: capitalize; vertical-align: middle; max-width:= 100%; margin-left: 0; margin-right: 0;" border=3D"0"></p>=09=09=09=09= =09=09=09=09=09</div>=0A=09=09=09=09=09=09=09=09=09<table border=3D"0" c= ellpadding=3D"0" cellspacing=3D"0" width=3D"100%" id=3D"template_contain= er" style=3D"background-color: #fff; border: 1px solid #dedede; box-shad= ow: 0 1px 4px rgba(0,0,0,.1); border-radius: 3px;" bgcolor=3D"#fff">=0A= =09=09=09=09=09=09=09=09=09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09<t= d align=3D"center" valign=3D"top">=0A=09=09=09=09=09=09=09=09=09=09=09= =09<!-- Header -->=0A=09=09=09=09=09=09=09=09=09=09=09=09<table border= =3D"0" cellpadding=3D"0" cellspacing=3D"0" width=3D"100%" id=3D"template= _header" style=3D'background-color: #3c3c3c; color: #fff; border-bottom:= 0; font-weight: bold; line-height: 100%; vertical-align: middle; font-f= amily: "Helvetica Neue",Helvetica,Roboto,Arial,sans-serif; border-radius= : 3px 3px 0 0;' bgcolor=3D"#3c3c3c">=0A=09=09=09=09=09=09=09=09=09=09=09= =09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09<td id=3D"header_= wrapper" style=3D"padding: 36px 48px; display: block;">=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09<h1 style=3D'font-family: "Helvetica Ne= ue",Helvetica,Roboto,Arial,sans-serif; font-size: 30px; font-weight: 300= ; line-height: 150%; margin: 0; text-align: left; text-shadow: 0 1px 0 #= 636363; color: #fff; background-color: inherit;' bgcolor=3D"inherit">Pas= sword Reset Request</h1>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09</t= d>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09= =09=09=09=09=09=09</table>=0A=09=09=09=09=09=09=09=09=09=09=09=09<!-- En= d Header -->=0A=09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09= =09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09<tr>=0A=09=09=09=09= =09=09=09=09=09=09=09<td align=3D"center" valign=3D"top">=0A=09=09=09=09= =09=09=09=09=09=09=09=09<!-- Body -->=0A=09=09=09=09=09=09=09=09=09=09= =09=09<table border=3D"0" cellpadding=3D"0" cellspacing=3D"0" width=3D"1= 00%" id=3D"template_body">=0A=09=09=09=09=09=09=09=09=09=09=09=09=09<tr>= =0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09<td valign=3D"top" id=3D"bo= dy_content" style=3D"background-color: #fff;" bgcolor=3D"#fff">=0A=09=09= =09=09=09=09=09=09=09=09=09=09=09=09=09<!-- Content -->=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09<table border=3D"0" cellpadding=3D"20"= cellspacing=3D"0" width=3D"100%">=0A=09=09=09=09=09=09=09=09=09=09=09= =09=09=09=09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09= =09<td valign=3D"top" style=3D"padding: 48px 48px 32px;">=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09=09=09=09<div id=3D"body_content_inner"= style=3D'color: #636363; font-family: "Helvetica Neue",Helvetica,Roboto= ,Arial,sans-serif; font-size: 14px; line-height: 150%; text-align: left;= ' align=3D"left">=0A=0A<p style=3D"margin: 0 0 16px;">Hi lee,</p>=0A<p s= tyle=3D"margin: 0 0 16px;">Someone has requested a new password for the= following account on The Whole Shebang:</p>=0A<p style=3D"margin: 0 0 1= 6px;">Username: lee</p>=0A<p style=3D"margin: 0 0 16px;">If you didn't m= ake this request, just ignore this email. If you'd like to proceed:</p>= =0A<p style=3D"margin: 0 0 16px;">=0A=09<a class=3D"link" href=3D"https:= //thewhole-shebang.co.uk/my-account/lost-password/?key=XXXX= XXXX&id=3D3" style=3D"font-weight: normal; text-decoration: underli= ne; color: #3c3c3c;">=09=09Click here to reset your password=09</a>=0A</= p>=0A=0A<p style=3D"margin: 0 0 16px;">Thanks for reading.</p>=0A=09=09= =09=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09</div>=0A=09=09=09=09=09= =09=09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09= =09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09= =09</table>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09=09<!-- End Cont= ent -->=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09= =09=09=09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09<= /table>=0A=09=09=09=09=09=09=09=09=09=09=09=09<!-- End Body -->=0A=09=09= =09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09=09</tr>= =0A=09=09=09=09=09=09=09=09=09</table>=0A=09=09=09=09=09=09=09=09</td>= =0A=09=09=09=09=09=09=09</tr>=0A=09=09=09=09=09=09=09<tr>=0A=09=09=09=09= =09=09=09=09<td align=3D"center" valign=3D"top">=0A=09=09=09=09=09=09=09= =09=09<!-- Footer -->=0A=09=09=09=09=09=09=09=09=09<table border=3D"0" c= ellpadding=3D"10" cellspacing=3D"0" width=3D"100%" id=3D"template_footer= ">=0A=09=09=09=09=09=09=09=09=09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09= =09<td valign=3D"top" style=3D"padding: 0; border-radius: 6px;">=0A=09= =09=09=09=09=09=09=09=09=09=09=09<table border=3D"0" cellpadding=3D"10"= cellspacing=3D"0" width=3D"100%">=0A=09=09=09=09=09=09=09=09=09=09=09= =09=09<tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09=09=09<td colspan=3D"2"= valign=3D"middle" id=3D"credit" style=3D'border-radius: 6px; border: 0;= color: #8a8a8a; font-family: "Helvetica Neue",Helvetica,Roboto,Arial,sa= ns-serif; font-size: 12px; line-height: 150%; text-align: center; paddin= g: 24px 0;' align=3D"center">=0A=09=09=09=09=09=09=09=09=09=09=09=09=09= =09=09<p style=3D"margin: 0 0 16px;">The Whole Shebang</p>=0A=09=09=09= =09=09=09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09=09= =09=09=09</tr>=0A=09=09=09=09=09=09=09=09=09=09=09=09</table>=0A=09=09= =09=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09=09=09=09</tr>= =0A=09=09=09=09=09=09=09=09=09</table>=0A=09=09=09=09=09=09=09=09=09<!--= End Footer -->=0A=09=09=09=09=09=09=09=09</td>=0A=09=09=09=09=09=09=09<= /tr>=0A=09=09=09=09=09=09</table>=0A=09=09=09=09=09</div>=0A=09=09=09=09= </td>=0A=09=09=09=09<td><!-- Deliberately empty to support consistent si= zing and layout across multiple email clients. --></td>=0A=09=09=09</tr>= =0A=09=09</table>=0A=09</body>=0A</html>=0A . 250 OK
The headers Postal captures are below - if it's something else you need, let me know and I'll try and get it.
As you mentioned, I can't capture the client headers (i.e from Outlook) and have a non-corrupted email. I can send the email via other means outside of Postal and capture that, though I'm not sure if that'll be of any use to you?
Lee
from postal.
catphish
commented on May 26, 2024
Can you pull the text out of the database manually? You mentioned you could read it there.
from postal.
pawprintdigital
commented on May 26, 2024
I can give you the message source, but it doesn't include the headers, just the body I'm afraid and that's what's posted above.
Postal is receiving and storing the message from the source fine it seems. It's only when it's then delivering the message/presenting for download it gets mangled.
I'm not sure which table Postal stores the header data in internally, but I imagine it'll just be as per the screenshot in my previous comment.
Any logging we can enable?
from postal.
pawprintdigital
commented on May 26, 2024
Great! If you need anything else, I'll be glad to help :)
from postal.
catphish
commented on May 26, 2024
We've now identified the problem. When the link tracking system modifies the email, we only save the updated message body, and ignore any changes to the headers.
In this case:
- The original body is encoded with "Quoted-printable" encoding (the weird
=0A=09=09=09=09=09=09=09=09=09=09stuff seen in the SMTP trace. - The modified version is not encoded, just plain HTML (as seen in your database).
- The headers still indicate that the message is quoted printable, but it's not. This causes it to appear mangled.
If you look at the raw .eml file in a text editor, you will see there is no corruption, just clean HTML, but a header saying it's quoted-printable, when it's not.
We will be working on a fix ASAP. In the meantime, you have two workarounds available:
- Send the emails as multipart with HTML and plain text (this is usually a good idea anyway)
- Disable click/view tracking so that the email is not rewritten.
Thanks to @adamcooke who did the hard work on reproducing this.
Fixed in 2834e2c
from postal.
pawprintdigital
commented on May 26, 2024
Hi @catphish, @adamcooke
Okay so that makes sense. We're using a Wordpress plugin on this website to generate the emails - if it's generating garbage headers then I'll file a bug with them too.
Unfortunately given the limitations of the plugin I'm unable to change the origin headers without butchering the plugin, which I'd like to avoid (as above, I'll raise a bug report with the plugin author and see if that can be resolved).
In the interim I've set Wordpress to just send the plain text version of this particular email so it's not a showstopper right now thankfully. I'm reluctant to disable the open tracking as it's come in handy a bunch of times when customers are complaining about missed deliveries etc, but it's good to know it's an option if this particular issue becomes more widespread in the interim.
I'd just like to thank you for your exceptional support with this. I raised the issue an hour ago (at 9pm no less) and you've already identified the issue, given me a workaround and are working on a permafix, which I think is absolutely incredible; even more so for an open source project that I can clearly see is a labour of love. I'd love to give something back as a token of my thanks, though I can't see a donate link or anything of the sort.
Thanks for all your help. I've been using Postal for a year now and it's been rock solid!
from postal.
pawprintdigital
commented on May 26, 2024
I assume it may take some time for this to filter down?
Lee
from postal.
pawprintdigital
commented on May 26, 2024
No worries - I'll try again in 15 and will let you know if the issue is resolved.
Thanks again
Lee
from postal.
pawprintdigital
commented on May 26, 2024
So I've got the new version installed, re-tested the email and this is now working flawlessly :)
from postal.
pawprintdigital
commented on May 26, 2024
There is one final thing..
After enabling click tracking, the reset link no longer works.
Appreciate this is a seperate issue (and I think there's another open issue surrounding & / & encoding), but for example..
Without click tracking, the output link is
https://thewhole-shebang.co.uk/my-account/lost-password/?key=XXXXX&id=3
With the click tracking, the resulting link is:
https://click.shop.thewhole-shebang.co.uk/nuknvj/XXXXX
Which directs to:
https://thewhole-shebang.co.uk/my-account/lost-password/?key=XXXXX&id=3
The broken & breaks the reset flow. It's not a major issue as I can live without the click tracking for the time being, but if you're currently in the god-mode fixing mood... ;)
Happy to create a new issue for this if I can't find an existing open issue for it.
Lee
from postal.
pawprintdigital
commented on May 26, 2024
Great, thanks.
I knew I'd seen it somewhere.
I'll leave this thread alone now as it's all resolved.
Thanks again both.
Lee
from postal.
Related Issues (20)
- Error when upgrading to 2.3.2 - Manifest unknow HOT 3
- Postal v3.3.2 SMTP server does not support TLS while using SMTP credentials
- Docker Network HOT 4
- Upgrade 2.3.2 : Powered by Postal unspecified. HOT 6
- Reverse proxy 403 forbidden error fix
- API /messages/message returns 500 if _expansions != array and doc is outdated
- Ip rotation not working HOT 1
- Message status not showing sent IP when sending using API (but does when sending using SMTP) HOT 3
- After upgrade to V3 no mails are sent HOT 6
- Local SMTP Endpoint Route Not Working HOT 5
- cannot load such file -- /opt/postal/app/lib/postal/version HOT 4
- ContainerConfig error at postal upgrade HOT 2
- Deleting an address endpoint causes not to be able to load routes anymore HOT 1
- V3: Storage is no longer emptied properly. HOT 7
- Outgoing mail spam check not working
- Errno::ENOENT: No such file or directory @ rb_sysopen - /config/signing.key
- Detected line with invalid line ending (missing <CR>) HOT 1
- The DNS record of the return path remains default and is not changing to the entry configured in the postal.yaml file.
- Private Mode Broken HOT 2
- BUG: routed emails not sent when ClamAV is enabled. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from postal.