emersion / hydroxide Goto Github PK

View Code? Open in Web Editor NEW

1.4K 39.0 119.0 425 KB

A third-party, open-source ProtonMail CardDAV, IMAP and SMTP bridge

License: MIT License

Go 100.00%

carddav protonmail smtp imap mail

hydroxide's People

Contributors

Stargazers

Watchers

Forkers

joshrsmith tcarrio pasdoy scelaris 0x00031337 agreenbhm philipbawn miguelramosfdz nast90210 sammyf directorx mondras icyphox cognition9144 chebyte backwardn thanson16 dvalter bicarus mikejzx jonathancross bonedaddy mikoim jcgoran msgpo jlarriba onemorebyte lazerl0rd almasry bertaland hubcapp it-zone wizbanicus mnixry antonme alitalia eddsalkield iot-resister x11x sbinet frezbo ineedthekeyboard kazazes proletarius101 midi-in tubbz-alt followmedown primalmotion icodein isgasho simhaonline the-blank-x enewbury matthewcroughan uionto johnjones4 j4nv5 sembeek tk-innoq laydros-forks fakeemperor learningasigo71 dcm eyedeekay iatenoodles eeveeboo eau-u4f cognitive-i suryatmodulus mattsains mrranmao brennoflavio baro77 ducksys netbospl yulqen mizzunet sebas05000 shrimpyuk 0800092000 patrick330602 iamricard gucio321 ink-splatters wonderfulshrinemaidenofparadise md7valentine battleb0t seraphn ydylla sukantag memst niedong macduff23 devpanther avalanchebreakdown iq-scm answer35 anubhavkini exokernel aetonsi

hydroxide's Issues

Cannot move messages/write into folders?

I can't seem to save a draft, save a copy to Sent (but maybe this appears to be done automatically when sending) or move items to folders (for example, to drafts or trash). I get this message:

2018/04/08 14:47:10 cannot handle update event for message : cannot get message sequence number in All Mail: message not found in local database

TLS & STARTTLS support

This will allow hydroxide to be hosted on a server and accessed from multiple remote clients.

'hydroxide serve' doesn't work

Hi there,
Here is what I obtain with 'serve' or 'server', any idea ?

max@machine:~$ hydroxide serve
2019/07/18 22:20:35 usage: hydroxide carddav
2019/07/18 22:20:35 usage: hydroxide smtp
2019/07/18 22:20:35 usage: hydroxide auth
2019/07/18 22:20:35 usage: hydroxide export-secret-keys
2019/07/18 22:20:35 usage: hydroxide status
2019/07/18 22:20:35 Unrecognized command

IMAP on Android

I'm running hydroxide on my Android phone locally using Go in Termux. I connect to it using the application K-9 mail. SMTP works great and sent messages arrive. However I'm unable to connect to the IMAP server. See attached screenshot (it says unable to connect to server in German):

Setting up IMAP with other mail applications doesn't work either, so I don't think this is an issue within K-9.

My IMAP settings:
Server: 127.0.0.1
Security: none
Port: 1143
Username: mail without "@protonmail.com"
Password: password supplied by hydroxide
IMAP-namespace: detect automatically

Can't build

I run Ubuntu 18.04, and have a fresh Go (1.12) install through snap.

Running go get github.com/emersion/hydroxide/cmd/hydroxide gives me the following output:

[15:39:33 last:0s ~] go get github.com/emersion/hydroxide/cmd/hydroxide
# github.com/emersion/hydroxide/smtp
go/src/github.com/emersion/hydroxide/smtp/smtp.go:32:19: cannot range over h.Header (type message.Header)
go/src/github.com/emersion/hydroxide/smtp/smtp.go:171:8: undefined: "github.com/emersion/go-message/mail".TextHeader
go/src/github.com/emersion/hydroxide/smtp/smtp.go:172:18: h.ContentType undefined (type "github.com/emersion/go-message/mail".PartHeader has no field or method ContentType)
go/src/github.com/emersion/hydroxide/smtp/smtp.go:186:3: impossible type switch case: p.Header (type "github.com/emersion/go-message/mail".PartHeader) cannot have dynamic type "github.com/emersion/go-message/mail".AttachmentHeader (Add method has pointer receiver)
# github.com/emersion/hydroxide/imap
go/src/github.com/emersion/hydroxide/imap/mailbox.go:318:19: invalid operation: h[key] (type message.Header does not support indexing)
go/src/github.com/emersion/hydroxide/imap/message.go:30:19: cannot range over h.Header (type message.Header)
go/src/github.com/emersion/hydroxide/imap/message.go:198:7: undefined: "github.com/emersion/go-message/mail".NewTextHeader
go/src/github.com/emersion/hydroxide/imap/message.go:209:7: undefined: "github.com/emersion/go-message/mail".NewAttachmentHeader
go/src/github.com/emersion/hydroxide/imap/message.go:235:7: undefined: "github.com/emersion/go-message/mail".NewHeader
go/src/github.com/emersion/hydroxide/imap/message.go:467:8: undefined: "github.com/emersion/go-message/mail".TextHeader
go/src/github.com/emersion/hydroxide/imap/message.go:468:18: h.ContentType undefined (type "github.com/emersion/go-message/mail".PartHeader has no field or method ContentType)
go/src/github.com/emersion/hydroxide/imap/message.go:482:3: impossible type switch case: p.Header (type "github.com/emersion/go-message/mail".PartHeader) cannot have dynamic type "github.com/emersion/go-message/mail".AttachmentHeader (Add method has pointer receiver)

Based on the absence of a hydroxide executable, I'm guessing the build failed. I suspect some updated libs, but I'm not sure. In issue #50, it is suggested to git clone the repo and build from that, to force some libs to specific versions I think, so I did. This gives me:

[15:43:03 last:0s ~/build] git clone [email protected]:emersion/hydroxide.git
[15:43:12 last:5s ~/build] cd hydroxide/
[15:43:16 last:0s ~/build/hydroxide] go build
go: finding github.com/emersion/go-textwrapper v0.0.0-20160606182133-d0e65e56babe
go: finding github.com/emersion/go-sasl v0.0.0-20161116183048-7e096a0a6197
go: finding github.com/emersion/go-smtp v0.11.0
go: finding github.com/emersion/go-imap-specialuse v0.0.0-20161227184202-ba031ced6a62
go: finding github.com/emersion/go-webdav v0.0.0-20180509190321-4ef680e9a32f
go: finding github.com/emersion/go-imap v1.0.0-beta.2
go: finding github.com/boltdb/bolt v1.3.1
go: finding golang.org/x/crypto v0.0.0-20190325154230-a5d413f7728c
go: finding golang.org/x/net v0.0.0-20190403144856-b630fd6fe46b
go: finding golang.org/x/text v0.3.0
go: finding github.com/emersion/go-bcrypt v0.0.0-20170822072041-6e724a1baa63
go: finding github.com/emersion/go-message v0.9.2
go: finding golang.org/x/sys v0.0.0-20190403152447-81d4e9dc473e
go: finding github.com/emersion/go-imap-move v0.0.0-20180601155324-5eb20cb834bf
go: finding golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a
go: finding github.com/howeyc/gopass v0.0.0-20170109162249-bf9dde6d0d2c
go: finding github.com/emersion/go-message v0.9.1
go: finding github.com/emersion/go-vcard v0.0.0-20190105225839-8856043f13c5
go: finding golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2
can't load package: package github.com/emersion/hydroxide: unknown import path "github.com/emersion/hydroxide": cannot find module providing package github.com/emersion/hydroxide

A bit strange, as I thought I was building that very package.

I hope you might have an idea of what I'm doing wrong.

IMAP & SMTP support

The official bridge isn't suited for power users.

Initial work for this is in the smtp branch.

Error on build: undefined: smtp.User

Hi all! I get an error when build hydroxide from the source:
go/src/github.com/emersion/hydroxide/smtp/smtp.go:352:54: undefined: smtp.User
go/src/github.com/emersion/hydroxide/smtp/smtp.go:373:38: undefined: smtp.User

I'm not good in Go, but it seems that instead of smtp.User smtp.Backend must be used?

Release tagging

Please consider tag releases so packagers can package your software.

void-linux/void-packages#6580 (comment)

554 Error: transaction failed, blame it on the weather: unknown sender address

This is the only result I get when trying to send anything. I tried with Mutt, plain msmtp, ClawsMail, sendEmail.

example (values in [] are hidden by me); hydroxide runs in background and reports nothing apart from Starting SMTP server at 127.0.0.1:1465:

$ sendEmail -f '[login]@protonmail.com' -t '[receiver]' -m 'test' -s localhost:1465 -l /tmp/smtp.log -xu '[login]' -xp '[password]' -vvv
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => Connecting to localhost:1465
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => My IP address is: 127.0.0.1
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 220 localhost ESMTP Service Ready
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Found SMTP success code: 220
Jan 09 13:24:21 localhost sendEmail[15925]: SUCCESS => Received: 	220 localhost ESMTP Service Ready
Jan 09 13:24:21 localhost sendEmail[15925]: INFO => Sending: 	EHLO localhost.localdomain
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 250-Hello localhost.localdomain
250-PIPELINING
250-8BITMIME
250 AUTH PLAIN
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Found SMTP success code: 250
Jan 09 13:24:21 localhost sendEmail[15925]: SUCCESS => Received: 	250-Hello localhost.localdomain
250-PIPELINING
250-8BITMIME
250 AUTH PLAIN
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => The remote SMTP server does NOT support TLS :(
Jan 09 13:24:21 localhost sendEmail[15925]: DEBUG => SMTP-AUTH: Using PLAIN authentication method
Jan 09 13:24:21 localhost sendEmail[15925]: INFO => Sending: 	AUTH PLAIN [authentication]
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 235 Authentication succeeded
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Found SMTP success code: 235
Jan 09 13:24:22 localhost sendEmail[15925]: SUCCESS => Received: 	235 Authentication succeeded
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => User authentication was successful (Method: PLAIN)
Jan 09 13:24:22 localhost sendEmail[15925]: INFO => Sending: 	MAIL FROM:<[login]@protonmail.com>
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 250 Roger, accepting mail from <[login]@protonmail.com>
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Found SMTP success code: 250
Jan 09 13:24:22 localhost sendEmail[15925]: SUCCESS => Received: 	250 Roger, accepting mail from <[login]@protonmail.com>
Jan 09 13:24:22 localhost sendEmail[15925]: INFO => Sending: 	RCPT TO:<[receiver]>
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 250 I'll make sure <[receiver]> gets this
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Found SMTP success code: 250
Jan 09 13:24:22 localhost sendEmail[15925]: SUCCESS => Received: 	250 I'll make sure <[receiver]> gets this
Jan 09 13:24:22 localhost sendEmail[15925]: INFO => Sending: 	DATA
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 354 Go ahead. End your data with <CR><LF>.<CR><LF>
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Found SMTP success code: 354
Jan 09 13:24:22 localhost sendEmail[15925]: SUCCESS => Received: 	354 Go ahead. End your data with <CR><LF>.<CR><LF>
Jan 09 13:24:22 localhost sendEmail[15925]: INFO => Sending message body
Jan 09 13:24:22 localhost sendEmail[15925]: Setting content-type: text/plain
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 554 Error: transaction failed, blame it on the weather: unknown sender address
Jan 09 13:24:22 localhost sendEmail[15925]: DEBUG => evalSMTPresponse() - Found SMTP error code: 554
Jan 09 13:24:22 localhost sendEmail[15925]: ERROR => Received: 	554 Error: transaction failed, blame it on the weather: unknown sender address

Do not save plaintext tokens

settings for servers

I'm not sure if I missed this in any of the documentation but is there a way to run the imap/smtp servers with custom settings for listen interface and port?

So far I've just been building my own version locally after editing the "port" and "addr" variables in hydroxide.go

Tutanota support possible?

would it be possible to add Tutanota support to this?
is their API similar enough to add support to this?
would a fork be better for supporting Tutanota?
or is Tutanota too different from Protonmail to re-use this codebase, which means it would be better to start something different from scratch to support Tutanota.

Pop345

Only SMTP? Where's POP?

Attachment Error 15212

Hi,
We have no problem to send mail through hydroxide, but as soon as we add an attachment, event a small txt file, we have this error:

[15212] Clé non valide (Invalid Key in english)

Apr 08 19:20:00 9c8e946c4438 sendEmail[1303]: DEBUG => evalSMTPresponse() - Checking for SMTP success or error status in the message: 554 Error: transaction failed, blame it on the weather: cannot send message: [15212] Clé non valide
Apr 08 19:20:00 9c8e946c4438 sendEmail[1303]: DEBUG => evalSMTPresponse() - Found SMTP error code: 554
Apr 08 19:20:00 9c8e946c4438 sendEmail[1303]: ERROR => Received: 	554 Error: transaction failed, blame it on the weather: cannot send message: [15212] Clé non valide

I'm doing my test with sendEmail but I have the same with a PHP Application and SwiftMailer

sendEmail -f '[email protected]' -t '[email protected]' -m 'mail content' -s localhost:1025 -l /tmp/smtp.log -xu '[email protected]' -xp 'my_bridge_password' -vvv -a /path/to/my/attachment_file.txt

Do you know what could cause this issue?
Thanks!

IMAP in bad state. "[401] Invalid access token".

Following is my log from hydroxide-imap running overnight with Thunderbird open and connected (unattended).

Nov 25 22:48:40 hydroxide[11625]: 2018/11/25 22:48:40 Synchronizing mailbox INBOX...
Nov 25 22:48:41 hydroxide[11625]: 2018/11/25 22:48:41 Synchronizing mailbox INBOX: done.
Nov 25 23:08:02 hydroxide[11625]: 2018/11/25 23:08:02 Synchronizing mailbox INBOX...
Nov 25 23:08:03 hydroxide[11625]: 2018/11/25 23:08:03 Synchronizing mailbox INBOX: done.
Nov 26 02:32:09 hydroxide[11625]: 2018/11/26 02:32:09 cannot receive event: [500] Internal server error
[snip ... same message every few seconds]
Nov 26 02:57:39 hydroxide[11625]: 2018/11/26 02:57:39 cannot receive event: [500] Internal server error
Nov 26 03:00:17 hydroxide[11625]: 2018/11/26 03:00:17 Synchronizing mailbox INBOX...
Nov 26 03:00:19 hydroxide[11625]: 2018/11/26 03:00:19 Synchronizing mailbox INBOX: done.
Nov 26 03:31:48 hydroxide[11625]: 2018/11/26 03:31:48 Received create event for message xyz==
Nov 26 06:54:47 hydroxide[11625]: 2018/11/26 06:54:47 Received update event for message xyz==
Nov 26 06:56:18 hydroxide[11625]: 2018/11/26 06:56:18 Received create event for message xyz==
Nov 26 07:41:18 hydroxide[11625]: 2018/11/26 07:41:18 Received create event for message xyz==
Nov 26 07:42:48 hydroxide[11625]: 2018/11/26 07:42:48 Received update event for message xyz==
Nov 26 07:43:47 hydroxide[11625]: 2018/11/26 07:43:47 Received update event for message xyz==
Nov 26 09:16:38 hydroxide[11625]: imap/server: 2018/11/26 09:16:38 cannot send response:  write tcp 127.0.0.1:7143->127.0.0.1:55100: write: broken pipe
Nov 26 09:16:51 hydroxide[11625]: 2018/11/26 09:16:51 Synchronizing mailbox INBOX...
Nov 26 09:16:52 hydroxide[11625]: 2018/11/26 09:16:52 Synchronizing mailbox INBOX: done.
Nov 26 09:17:51 hydroxide[11625]: 2018/11/26 09:17:51 cannot receive event: cannot re-authenticate: two factor authentication enabled, please login manually
Nov 26 09:17:51 hydroxide[11625]: 2018/11/26 09:17:51 cannot receive event: [401] Invalid access token
[snip ... same message multiple times per second]
Nov 26 10:21:04 hydroxide[11625]: 2018/11/26 10:21:04 cannot receive event: [401] Invalid access token
Nov 26 10:21:04 systemd[793]: Stopping Hydroxide IMAP proxy....
Nov 26 10:21:04 systemd[793]: Stopped Hydroxide IMAP proxy..
Nov 26 10:21:04 systemd[793]: Started Hydroxide IMAP proxy..
Nov 26 10:21:04 hydroxide[22743]: 2018/11/26 10:21:04 Starting IMAP server at 127.0.0.1:7143
Nov 26 10:21:20 hydroxide[22743]: 2018/11/26 10:21:20 Synchronizing mailbox INBOX...
Nov 26 10:21:21 hydroxide[22743]: 2018/11/26 10:21:21 Synchronizing mailbox INBOX: done.

So it thinks it needs to re-authenticate due to MFA, but restarting the service causes it to work again (without re-authentication).

The log also shows a period of 500 errors (I guess that is the PM server?), but hydroxide recovers from that.

IMAP server does not support multiple connections

As @v01d mentioned in #23, IMAP server in Hydroxide does support only one active connection per account. Any but first connection does not progress to AUTHENTICATE completed usless previous one would be closed.

This is crucial (and worth documenting at very least) due to Thunderbird (and proboably some other clients) by default use multiple connections (5 in my case), which made hydroxide usage virtually impossible due to

I'm trying hydroxide with Thunderbird and I noticed that I can only access a folder, after which clicking on other folders does not work.

Steps to reproduce

(Optionnaly) start traffic recording with tcpdump/wireshark/whatever on loopback interface.
Open IMAP socket and login as <user>@protonmail.com
Repeat the same without closing previous connection via another instance of any client or simply telnet
It will stuck at AG client request

Fix for Thunderbird

Go to Account Setting (via Edit, or "Burger"->Preferences)
Click Server Settings for the Protonmail account
Click Advanced... button
Set Maximum number of server connections to cache to 1

I hope this issue at least will help someone to sort out "Connection to server 127.0.0.1 timed out" errors in Thunderbird.

imap: subscriptions support

Start multiple servers

Start multiple servers at the same time, sharing the same user session.

openpgp unsupported feature

hydroxide auth … results in openpgp: unsupported feature: public key algorithm 22

smtp: sign messages

The latest API requires outgoing messages to be signed.

Receive events

Hide password when typing

Password is not hidden when running hydroxide auth.

Hydroxide build / installation errors

I get the following error output when I follow the installation instructions outlined in the readme:

$ go get github.com/emersion/hydroxide/cmd/hydroxide
# github.com/emersion/go-smtp
go/src/github.com/emersion/go-smtp/client.go:188: config.Clone undefined (type *tls.Config has no field or method Clone)
# github.com/emersion/go-webdav/carddav
go/src/github.com/emersion/go-webdav/carddav/carddav.go:84: r.Context undefined (type *http.Request has no field or method Context)
# github.com/emersion/go-message/textproto
go/src/github.com/emersion/go-message/textproto/header.go:347: undefined: strings.Builder
go/src/github.com/emersion/go-message/textproto/header.go:364: undefined: strings.Builder
# github.com/emersion/go-imap/client
go/src/github.com/emersion/go-imap/client/client.go:645: tlsConfig.Clone undefined (type *tls.Config has no field or method Clone)
go/src/github.com/emersion/go-imap/client/cmd_noauth.go:42: tlsConfig.Clone undefined (type *tls.Config has no field or method Clone)
# github.com/emersion/hydroxide/protonmail
go/src/github.com/emersion/hydroxide/protonmail/protonmail.go:102: req.GetBody undefined (type *http.Request has no field or method GetBody)
go/src/github.com/emersion/hydroxide/protonmail/protonmail.go:121: req.GetBody undefined (type *http.Request has no field or method GetBody)
go/src/github.com/emersion/hydroxide/protonmail/protonmail.go:130: req.GetBody undefined (type *http.Request has no field or method GetBody)
# golang.org/x/sys/unix
go/src/golang.org/x/sys/unix/ioctl.go:18: undefined: runtime.KeepAlive
go/src/golang.org/x/sys/unix/ioctl.go:28: undefined: runtime.KeepAlive
go/src/golang.org/x/sys/unix/syscall_linux.go:100: undefined: runtime.KeepAlive

$ go version
go version go1.6.2 linux/amd64

Allow not to store passwords on disk

This may need frequent re-authentication on the server.

Support multiple email addresses

Protonmail Plus and on allow you to have multiple email accounts under one account. As far as I know it is to be used with your own domain.

eg. [email protected] and [email protected]

Is this currently supported by hydroxide? From Thunderbird I can see that openpgp does not read the rest of the keys to decrypt the message.

Although I can see the list of messages when I try to open a message from a different email eg [email protected] then I get the following error: The current operation on Inbox did not succeeded. The mail server for account [email protected] responded: openpgp: incorrect key.

I think each email has it's own key pair.

Login failure for IMAP and SMTP

Hi,

I am trying to setup the client to connect to the IMAP and the SMTP server but I have not been successful. I tried both thunderbird and apple's mail.app. I tried through the terminal as well but I had no luck. I tried to look at the source code to see if I could figure things out but no luck there either.

I get the following error: Verify that the username and password are correct.

I managed to connect to the smtp without authentication but when I try to send an email it doesn't let me.

So here is what I have done

hydroxide auth MYUSERNAME
Password:
Bridge password:  base64__32_byte_password= 
#  I use the string above as my password

On different terminals I load imap and smtp

hydroxide imap
hydroxide smtp

I can use nc to connect to both servers

# IMAP
 nc 127.0.0.1 1143
* OK [CAPABILITY IMAP4rev1 AUTH=PLAIN] IMAP4rev1 Service Ready

# SMTP
nc 127.0.0.1 1025
220 localhost ESMTP Service Ready
ehlo localhost
250-Hello localhost
250-PIPELINING
250-8BITMIME
250 AUTH PLAIN

For the client set up I use the following:
IMAP

Username:  USERNAME (without @protonmail.com)
Password:  base64__32_byte_password= (I tried decoding the base64 and pasting that but it didn't work either)
Hostname: localhost or 127.0.0.1 (tried both)
Port: 1143
Authentication: Password
Allow insecure authentication: True
IMAP Path Prefix: empty
TLS Certificate: None

SMTP

Username: USERNAME (without @protonmail.com)
Password:   base64__32_byte_password= (I tried decoding the base64 and pasting that but it didn't work either)
Hostname: localhost or 127.0.0.1 (tried both)
Port: 1025
Authentication: Password or None (None connects but I can't send an email)

I fought for hours trying to make it work, I would appreciate some help. Once I figure it out I can write up a guide on how to set it up. Perhaps you can add it to the repo for other people to access.

Thank you for your help.

Cannot authenticate

I’ve just noticed that hydroxide auth results in error:

./hydroxide auth {here goes username}   
Password: {here goes hidden password}
2018/03/03 17:59:34 invalid character '<' looking for beginning of value

Instead of the error Hydroxide should ask for 2fa code.

Check SRP modulus signature

See #46 (comment)

Need to check the signature (including the signature's keys).

Support signed outgoing messages

ProtonMail is capable of sending PGP-signed messages (tested with Android app). However signed messages sent with Hydroxide get mangled on the way.

As a result:

signatures are not recognised by MUAs (tested with K-9 Mail and NeoMutt)
manual verification results in BAD signature.

Example:
What was sent by MUA to Hydroxide’s SMTP server (sniffed with Wireshark):

From: {email address}     
To: {email address}
Subject: sign test watched
Message-ID: <[email protected]>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="p4hhyhz6uanwonvv"
Content-Disposition: inline
User-Agent: NeoMutt/20180716


--p4hhyhz6uanwonvv
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

sign test
--=20
Adam Pioterek

--p4hhyhz6uanwonvv
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEElPPQXqHLfrlrrE+/JELid3bgpXgFAltTbBwACgkQJELid3bg
pXiTugf/bf8dHUtmfOHgFe4rumOcTf1rw4goQX/xK4lBpVuaxrVEflNyeWY40s6J
6Ah8Om9IQZ7WCxs7s76D+8QQVCPmjrrKLBW71qJY2eUr2wZREZoTF4BY3N9flRSV
YOiMm/UQuyO6s15B5YROq+o/0SXMEKxx56bbprRIWS7lXJYC0LQyNLgUliB8btL1
eMjSQBwZ+2nf0FxIdTdXwe6bpYq8/dNT6ZouvlYsjeyjUUyrEXCiTbacM3Wyk/MR
MpxUUmbTdZpFNKKnGpIpZld7oGU80bM3Qty+huuPP3UIfXckjwn/YZ4vn1aZsu01
C4r/MuCeLfxa6TJ6U8YDj5BWYTyqMw==
=slx2
-----END PGP SIGNATURE-----

--p4hhyhz6uanwonvv--

What was sent by ProtonMail (what lies in my Sent folder):

X-Pm-Origin: internal
X-Pm-Content-Encryption: end-to-end
Subject: sign test watched
To: {email address}
From: Adam Pioterek <{email address}>
Date: Sat, 21 Jul 2018 13:23:56 -0400
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="---------------------5431deeefc95158512b875cffd20fe39"
X-Attached: signature.asc
Message-Id: <7VlPHNPCzHlNg4x3Od1sq0WvVWLGVWR9PVpda_aEiPMP0eQsdxQnTwT41QDWYEBfHwgndTUeI7P2zt-ekzIH_-lYD5336rAq_NnVeYNM2XE=@protonmail.ch>
X-Pm-Recipient-Authentication: {email address}=none
X-Pm-Recipient-Encryption: {email address}=none


-----------------------5431deeefc95158512b875cffd20fe39
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;charset=utf-8

sign test
-- =


Adam Pioterek

-----------------------5431deeefc95158512b875cffd20fe39
Content-Type: application/pgp-signature; name="signature.asc"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="signature.asc"

LS0tLS1CRUdJTiBQR1AgU0lHTkFUVVJFLS0tLS0KCmlRRXpCQUVCQ0FBZEZpRUVsUFBRWHFITGZy
bHJyRSsvSkVMaWQzYmdwWGdGQWx0VGJCd0FDZ2tRSkVMaWQzYmcKcFhpVHVnZi9iZjhkSFV0bWZP
SGdGZTRydW1PY1RmMXJ3NGdvUVgveEs0bEJwVnVheHJWRWZsTnllV1k0MHM2Sgo2QWg4T205SVFa
N1dDeHM3czc2RCs4UVFWQ1BtanJyS0xCVzcxcUpZMmVVcjJ3WlJFWm9URjRCWTNOOWZsUlNWCllP
aU1tL1VRdXlPNnMxNUI1WVJPcStvLzBTWE1FS3h4NTZiYnByUklXUzdsWEpZQzBMUXlOTGdVbGlC
OGJ0TDEKZU1qU1FCd1orMm5mMEZ4SWRUZFh3ZTZicFlxOC9kTlQ2Wm91dmxZc2pleWpVVXlyRVhD
aVRiYWNNM1d5ay9NUgpNcHhVVW1iVGRacEZOS0tuR3BJcFpsZDdvR1U4MGJNM1F0eStodXVQUDNV
SWZYY2tqd24vWVo0dm4xYVpzdTAxCkM0ci9NdUNlTGZ4YTZUSjZVOFlEajVCV1lUeXFNdz09Cj1z
bHgyCi0tLS0tRU5EIFBHUCBTSUdOQVRVUkUtLS0tLQo=
-----------------------5431deeefc95158512b875cffd20fe39--

blame it on the weather: sender address key hasn\'t been decrypted'

Error in SMTP using nodemailer. Checked in the code. Looks like the private key is null after decryption..

Anything I could do to fix this?

Something goes wrong in this code

var privateKey *openpgp.Entity for _, e := range u.privateKeys { if e.PrimaryKey.KeyId == encryptedPrivateKey.PrimaryKey.KeyId { privateKey = e break } } if privateKey == nil { return errors.New("sender address key hasn't been decrypted") }

Here's my nodemailer config if it can help.

let transporterSale= nodemailer.createTransport({ host:'127.0.0.1', port:1025, secure:false, auth:{user:"USER",pass:"BRIDGEPW"}, tls:{rejectUnauthorized:false} })

Merci

smtp: support non-primary addresses

Only primary addresses are supported right now.

How to build your go project in a sandboxed manner?

Hello,

I'm maintaining the arch linux / AUR package for hydroxide-git package and so far I was able to build it using these instructions inside a chroot:

build() {
	GOPATH="$srcdir" go get -fix -v -x ${_gourl}/cmd/hydroxide
}

check() {
	GOPATH="$GOPATH:$srcdir" go test -v -x ${_gourl}/cmd/hydroxide
}

package() {

	mkdir -p "$pkgdir/usr/bin"
	install -p -m755 "$srcdir/bin/"* "$pkgdir/usr/bin"

	mkdir -p "$pkgdir/usr/lib/go"
	cp -Rv --preserve=timestamps "$srcdir/src" "$pkgdir/usr/lib/go"

	install -Dm644 "$srcdir/src/$_gourl/LICENSE" "$pkgdir/usr/share/licenses/$pkgname/LICENSE"

}

Since your last updates (around 2 weeks ago) I'm trying to figure out how to build hydroxide in this sandboxed manner, but I'm unable to understand WTF is going on behind the scenes.

I do not want to pollute the user's machine. Which means both GOROOT and GOPATH have to be inside the chroot, and NOT a global one. That's indiscussable and is a fixed requirement.

Now the PKGBUILD file looks like this:

build() {
	export GOPATH="$srcdir/gopath";
	export GOROOT="$srcdir/goroot";

	cd "$srcdir";
	go get -v -x ${_gourl}/cmd/hydroxide;

}

check() {

	export GOPATH="$srcdir/gopath";
	export GOROOT="$srcdir/goroot";

	cd "$srcdir";
	go test -v -x ${_gourl}/cmd/hydroxide;

}

When I set the GOPATH and GOROOT I will end up with "import path does not begin with hostname" errors everywhere now. Literally, thousands of packages in the dependency tree seem to use the legacy syntax before go 1.7. Google Fu returns only automated build pipeline errors, and not a single explanation on how the problem came to be and how to solve it. People always say "just unset your GOPATH or GOROOT, and it will work" - which I cannot do because of the chroot.

So, my questions are now something like this:

Why can't I do something like git clone hydroxide ./somewhere; cd ./somewhere; go build;?
Do I have to use go get? I want to avoid it, because go get is too stupid for a package cache, and will always pollute stuff outside the goroot, in multiple ways. And it seems that go get also is too stupid to resolve the namespace of the package it is currently building; with no instructions searchable on how to fix that either.
How to build hydroxide without using go get?

PS: Please, bear with me, I'm not a go developer, so I am just trying to get shit to run; and assume that I'm a noob level zero.

Verify signature, decrypt contacts

Attachments missing signatures

From time to time (but it happens only when there’s at least 1 attachment; i.e. apart from message body) I get the following error:

msmtp: the server did not accept the mail
msmtp: server message: 554 Error: transaction failed, blame it on the weather:
cannot send message: [15197] One or more attachments is missing a signature
msmtp: could not send mail (account {account name} from {config file})

Implement authentication

Messages misidentification

After pulling from ProtonMail local mailbox is not the same as remote (that on PM servers).

Messages that were deleted in PM web client are still present in the local mailbox (others are deleted), on new mail in Inbox there are sometimes messages that were already read but are once again marked unread (from the same thread or other), archiving in web client actually archives different messages.

Dockerfile

Thank you for this project ! I really don't think to use the proprietary bridge from protonmail.

Since hydroxide is meant to run headless, I think it makes sense to have a small dockerfile for
it and run it in a separate container. Below, there is a simple example to build and create such container.

To make it usable I need a few more information.

For carddav, do I still need a reverse proxy or I can connect directly to the port offered by hydroxide
?
Where the configuration files are stored ? Docker can bind mount a directory (ex. ~/.hydroxide) or create a docker volume to store all the configuration file and persistent data.
At the moment if I want smtp, imap and carddav, I need to launch three process. Maybe you can change the command line to select which services should be available and then fork three threads ? Something like :

> docker run hydroxide carddav smtp
2019/02/27 08:28:27 Starting CardDAV server at 127.0.0.1:8080
2019/02/27 08:25:33 Starting SMTP server at 127.0.0.1:1025

Otherwise to containerize hydroxide I need to add a process manager that defeats a bit the philosophy of one process per container ...

Once you have docker installed on your machine and saved the snipped below as Dockerfile , it's just as simple as docker build . -t hydroxide and docker run hydroxide

FROM golang:1-alpine as builder
  
RUN apk --update upgrade \
&& apk --no-cache --no-progress add git make gcc musl-dev \
&& rm -rf /var/cache/apk/*

RUN go get github.com/emersion/hydroxide/cmd/hydroxide

FROM alpine:3.9
RUN apk --update upgrade \
    && apk --no-cache --no-progress add ca-certificates git \
    && rm -rf /var/cache/apk/*

EXPOSE 1025
EXPOSE 1143
EXPOSE 8080

COPY --from=builder /go/bin/hydroxide /usr/bin/hydroxide

ENTRYPOINT ["/usr/bin/hydroxide"]

And one last question : this is me not understanding... when I do hydroxide auth <username> which username is that ? protonmail ? Or a local username used to store all the protonmail data separate from others ?

Auth error: "failed to unlock key [key]. openpgp: invalid data: private key checksum failure

Hello,
I'm seeing from #29 that I should use my normal ProtonMail password when trying to set up credentials with hydroxide auth. However, I get this error every time:

./hydroxide auth [username]
Password:
2018/11/24 10:27:41 failed to unlock key [key]: openpgp: invalid data: private key checksum failure
Bridge password: rBLy+TpeewzrvoEN+jJie0Yh9409IHPDauvSF0AsbRM=

Am I doing something wrong?
Thanks,
Daniel Hines

Cannot authorise, openpgp error

I seem to have a problem authorising, despite the fact that use the right credentials. Can you help out with the following?

hydroxide auth username
Password:
Mailbox password:
2018/01/29 09:42:16 openpgp: invalid data: private key checksum failure

Thanks!

High CPU usage by hydroxide imap when offline

When no connection to Protonmail is available (e.g. computer wakes up without Internet connection), hydroxide may start doing the following queries a few thousand times a second:
hydroxide[1325]: 2019/05/23 14:37:41 cannot receive event: Get https://dev.protonmail.com/api/events/u<...>A==: dial tcp: lookup dev.protonmail.com: Temporary failure in name resolution

Some sort of timeout (probably progressive) may fix this.

CardDAV in Thunderbird/CardBook

CardDav doesn't seem to work in Thunderbird using the CardBook add-on. It authenticates and detects the address book "ProtonMail", but no contacts are synchronized. This is the last error I see in the log.

Nov 25 22:03:51 hydroxide[10431]: Method Not Allowed2018/11/25 22:03:51 ERROR &{PROPFIND /.well-known/carddav HTTP/1.1 1 1 map[Content-Type:[application/xml; charset=utf-8] Authorization:[Basic xxx] Content-Length:[122] User-Agent:[Thunderbird CardBook/33.2 Lightning/6.2.3.1] Accept:[*/*] Accept-Language:[en-US,en;q=0.5] Accept-Encoding:[gzip, deflate] Depth:[0] Connection:[keep-alive]] 0xc0000b4180 <nil> 122 [] false localhost:7080 map[] map[] <nil> map[] 127.0.0.1:57038 /.well-known/carddav <nil> <nil> <nil> 0xc0000b4280} carddav: not found

Am I correct to enter just the base URL? (http://127.0.0.1:7080).

imap: expose custom labels/folders as mailboxes

Implement read-write addressbook

Add a padding to encrypted saved credentials

To make it more difficult to guess the password length.

Use memguard to store secrets in memory

See https://github.com/awnumar/memguard

Which ProtonMail plan do I need?

Do I still need one of paid plans (enabled IMAP/SMP support in my account) to use this?

Add a timeout for sessions

Currently, once opened sessions never die.

Documentation typo?

The README.md indicates to use this to install: go get github.com/emersion/hydroxide but that results in "can't load package: package github.com/emersion/hydroxide: no Go files in /home/josh/go/src/github.com/emersion/hydroxide"

I think it should say: go get github.com/emersion/hydroxide/cmd/hydroxide ?

ProtonMail requires new version

Hydroxide reports
2018/10/16 19:45:56 [5003] Update required: Please refresh the page to load the latest version of the application
upon login.

Document how to use/install

Hi, just found this and I'm really interested in it. However, I don't really understand how it works (I know it talks to protonmail's using its API). For example, does it retrieve the private key on its own from the server? The password for the key (the one used for login) is the password that should be used for the auth command?
I would like also to understand how is the sensitive data stored (key, password, decrypted emails, etc).