stalwartlabs / mail-server Goto Github PK

Really liking the look of this server, but it would be great if more SQL flavors than SQLite were supported for storing the bitmaps/logs/etc.
In my case I'd love the ability to link the server to a postgres database.

Which feature or improvement would you like to request?

It would be great, if the "All Mail" folder could be disabled/hidden. I'm not sure, if it's maybe technically necessary for something (JMAP?), but it just bothers me, because it's there and I don't need it.
Also because of how my mail client syncs via IMAP the "All Mail" folders read counter is always behind. So I read a mail in the inbox the read count in the inbox goes down and then it takes a few second until the read count for the "All Mail" folder also goes down. The last thing of course isn't the fault of Stalwart, but it just doubles my annoyance about the folder being there in the first place.

Is your feature request related to a problem?

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

When you have a folder with a & you can't use them in various mail clients.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

1. Connect Blue Mail
2. Make a mailbox that contains a & like: You & me
3. Try to move a file into the folder
4. The folder stays empty

Version

v0.3.7

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

LDAP

What operating system are you using?

Linux

Relevant log output

2023-09-16T07:39:00.353411Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::client: event="read" data="22 NOOP\r\n" size=9
2023-09-16T07:39:00.353768Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::writer: event="write" data="22 OK NOOP completed\r\n" size=22
2023-09-16T07:39:00.356338Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::client: event="read" data="23 SELECT \"INBOX\"\r\n" size=19
2023-09-16T07:39:00.356749Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::writer: event="write" data="* OK [CLOSED] Closed previous mailbox\r\n* 3 EXISTS\r\n* FLAGS (\\Answered \\Flagged \\Deleted \\Seen \\Draft)\r\n* 0 RECENT\r\n* OK [PERMANENTFLAGS (\\Deleted \\Seen \\Answered \\Flagged \\Draft \\*)] All allowed\r\n* OK [UIDVALIDITY 1694717808] UIDs valid\r\n* OK [UIDNEXT 5] Next predicted UID\r\n* OK [MAILBOXID (cyaaaaaa)] Unique Mailbox ID\r\n23 OK [READ-WRITE] SELECT completed\r\n" size=359
2023-09-16T07:39:00.389898Z  INFO session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: utils::listener::listen: context="tls" event="handshake" version=TLSv1_3 cipher=TLS13_AES_256_GCM_SHA384
2023-09-16T07:39:00.393480Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::client: event="read" data="1 LOGIN \"[email protected]\" \"password\"\r\n" size=47
2023-09-16T07:39:00.397251Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::writer: event="write" data="1 OK [CAPABILITY IMAP4rev2 IMAP4rev1 ENABLE SASL-IR LITERAL+ ID UTF8=ACCEPT IDLE NAMESPACE CHILDREN MULTIAPPEND BINARY UNSELECT ACL UIDPLUS ESEARCH WITHIN SEARCHRES SORT THREAD=REFERENCES LIST-EXTENDED ESORT SORT=DISPLAY SPECIAL-USE CREATE-SPECIAL-USE MOVE CONDSTORE QRESYNC UNAUTHENTICATE STATUS=SIZE OBJECTID PREVIEW] Authentication successful\r\n" size=347
2023-09-16T07:39:00.399752Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::client: event="read" data="2 STATUS \"test email you &- me\" (UIDVALIDITY)\r\n" size=47
2023-09-16T07:39:00.400256Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::writer: event="write" data="* STATUS \"test email you &- me\" (UIDVALIDITY 1694718505)\r\n2 OK STATUS completed\r\n" size=81
2023-09-16T07:39:00.401881Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::client: event="read" data="3 NOOP\r\n" size=8
2023-09-16T07:39:00.401977Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::writer: event="write" data="3 OK NOOP completed\r\n" size=21
2023-09-16T07:39:00.404276Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::client: event="read" data="24 STATUS \"test email you &- me\" (RECENT)\r\n" size=43
2023-09-16T07:39:00.404382Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::writer: event="write" data="* STATUS \"test email you &- me\" (RECENT 0)\r\n24 OK STATUS completed\r\n" size=68
2023-09-16T07:39:00.408752Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::client: event="read" data="25 UID COPY 3 \"test email you &- me\"\r\n" size=38
2023-09-16T07:39:00.408879Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::writer: event="write" data="25 NO [TRYCREATE] Destination mailbox does not exist.\r\n" size=55
2023-09-16T07:39:00.422153Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::client: event="read" data="4 NOOP\r\n" size=8
2023-09-16T07:39:00.422240Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::writer: event="write" data="4 OK NOOP completed\r\n" size=21
2023-09-16T07:39:00.425144Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::client: event="read" data="5 EXAMINE \"test email you &- me\"\r\n" size=34
2023-09-16T07:39:00.425499Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::writer: event="write" data="* 0 EXISTS\r\n* FLAGS (\\Answered \\Flagged \\Deleted \\Seen \\Draft)\r\n* 0 RECENT\r\n* OK [PERMANENTFLAGS (\\Deleted \\Seen \\Answered \\Flagged \\Draft \\*)] All allowed\r\n* OK [UIDVALIDITY 1694718505] UIDs valid\r\n* OK [UIDNEXT 1] Next predicted UID\r\n* OK [MAILBOXID (cyaaaaaf)] Unique Mailbox ID\r\n5 OK [READ-ONLY] EXAMINE completed\r\n" size=319
2023-09-16T07:39:00.430804Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::client: event="read" data="6 SEARCH RETURN (COUNT) 1:* UNSEEN NOT DELETED\r\n" size=48
2023-09-16T07:39:00.430987Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=37634}: imap::core::writer: event="write" data="* ESEARCH (TAG \"6\") COUNT 0\r\n6 OK SEARCH completed\r\n" size=52
2023-09-16T07:39:00.447501Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::client: event="read" data="26 NOOP\r\n" size=9
2023-09-16T07:39:00.447684Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::writer: event="write" data="26 OK NOOP completed\r\n" size=22
2023-09-16T07:39:00.449729Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::client: event="read" data="27 EXAMINE \"INBOX\"\r\n" size=20
2023-09-16T07:39:00.450123Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::writer: event="write" data="* OK [CLOSED] Closed previous mailbox\r\n* 3 EXISTS\r\n* FLAGS (\\Answered \\Flagged \\Deleted \\Seen \\Draft)\r\n* 0 RECENT\r\n* OK [PERMANENTFLAGS (\\Deleted \\Seen \\Answered \\Flagged \\Draft \\*)] All allowed\r\n* OK [UIDVALIDITY 1694717808] UIDs valid\r\n* OK [UIDNEXT 5] Next predicted UID\r\n* OK [MAILBOXID (cyaaaaaa)] Unique Mailbox ID\r\n27 OK [READ-ONLY] EXAMINE completed\r\n" size=359
2023-09-16T07:39:00.455586Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::client: event="read" data="28 SEARCH RETURN (COUNT) 1:* UNSEEN NOT DELETED\r\n" size=49
2023-09-16T07:39:00.455864Z TRACE session{instance="imaptls" protocol=Imap remote.ip="::ffff:11.11.11.11" remote.port=46664}: imap::core::writer: event="write" data="* ESEARCH (TAG \"28\") COUNT 0\r\n28 OK SEARCH completed\r\n" size=54

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

What happened?

When I get certain e-mails over IMAP the encoding of certain non ASCII characters (like German umlauts) is messed up. It for example shows "Anh�nge" instead of "Anhänge". When I get the same e-mail over an JMAP client everything is fine.
It's just happening with certain e-mails and not with most e-mails, but because the e-mail I got is an automated, regular one and was always showing correctly before using Stalwart, I can be pretty certain it's actually somehow related to the mail server.

How can we reproduce the problem?

I'm not completely sure, but my best guess is that is has something to do with the e-mail encoding. The e-mail it occurred with is a multipart e-mail with "Content-Type: text/plain; charset=iso-8859-1"/"Content-Type: text/html; charset=iso-8859-1" and "Content-Transfer-Encoding: 8bit". I could privately provide the whole e-mail file out of the mail dir.

Version

v0.3.x

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

SQLite

What operating system are you using?

Linux

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

I'm using the Docker container with an SQL directory and a local filesystem blob storage. I'm trying to migrate large mailbox using Isync. When I do so, it works briefly then every mail results in a line like this in the logs:

2023-07-21T00:26:02.191751Z ERROR imap::core::message: Failed to store uid map event="error" context="store" account_id=0 collection=Mailbox error=AssertValueFailed

Retrying results in failure. Removing the contents of the data directory and retrying results in failure.

Which feature or improvement would you like to request?

Remove the syslog options: systemd[1]: /etc/systemd/system/stalwart-mail.service:16: Standard output type syslog is obsolete, automatically updating to journal. Please update your unit file, and consider removing the setting altogether.

Is your feature request related to a problem?

I'm always frustrated when...

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

Stdout in cmd.exe looks like

←[2m2023-08-13T15:57:41.338954Z←[0m ←[32m INFO←[0m ←[2mutils←[0m←[2m:←[0m Starting Stalwart Mail Server v0.3.3...
←[2m2023-08-13T15:57:41.394653Z←[0m ←[34mDEBUG←[0m ←[2mjmap::services::housekeeper←[0m←[2m:←[0m Housekeeper task started.
←[2m2023-08-13T15:57:41.394870Z←[0m ←[32m INFO←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Starting listener ←[3mid←[0m←[2m=←[0m"imaptls" ←[3mprotocol←[0m←[2m=←[0mImap ←[3mbind.ip←[0m←[2m=←[0m"::" ←[3mbind.port←[0m←[2m=←[0m993 ←[3mtls←[0m←[2m=←[0mtrue
←[2m2023-08-13T15:57:41.397053Z←[0m ←[32m INFO←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Starting listener ←[3mid←[0m←[2m=←[0m"jmap" ←[3mprotocol←[0m←[2m=←[0mJmap ←[3mbind.ip←[0m←[2m=←[0m"::" ←[3mbind.port←[0m←[2m=←[0m8080 ←[3mtls←[0m←[2m=←[0mfalse
←[2m2023-08-13T15:57:41.397929Z←[0m ←[32m INFO←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Starting listener ←[3mid←[0m←[2m=←[0m"sieve" ←[3mprotocol←[0m←[2m=←[0mManageSieve ←[3mbind.ip←[0m←[2m=←[0m"::" ←[3mbind.port←[0m←[2m=←[0m4190 ←[3mtls←[0m←[2m=←[0mtrue
←[2m2023-08-13T15:57:41.398584Z←[0m ←[32m INFO←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Starting listener ←[3mid←[0m←[2m=←[0m"smtp" ←[3mprotocol←[0m←[2m=←[0mSmtp ←[3mbind.ip←[0m←[2m=←[0m"::" ←[3mbind.port←[0m←[2m=←[0m25 ←[3mtls←[0m←[2m=←[0mfalse
←[2m2023-08-13T15:57:41.399198Z←[0m ←[32m INFO←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Starting listener ←[3mid←[0m←[2m=←[0m"submissions" ←[3mprotocol←[0m←[2m=←[0mSmtp ←[3mbind.ip←[0m←[2m=←[0m"::" ←[3mbind.port←[0m←[2m=←[0m465 ←[3mtls←[0m←[2m=←[0mtrue
←[2m2023-08-13T15:57:42.743375Z←[0m ←[32m INFO←[0m ←[2mutils←[0m←[2m:←[0m Shutting down Stalwart Mail Server v0.3.3...
←[2m2023-08-13T15:57:42.743794Z←[0m ←[34mDEBUG←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Listener shutting down. ←[3mevent←[0m←[2m=←[0m"shutdown" ←[3minstance←[0m←[2m=←[0m"sieve" ←[3mprotocol←[0m←[2m=←[0mManageSieve
←[2m2023-08-13T15:57:42.743892Z←[0m ←[34mDEBUG←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Listener shutting down. ←[3mevent←[0m←[2m=←[0m"shutdown" ←[3minstance←[0m←[2m=←[0m"smtp" ←[3mprotocol←[0m←[2m=←[0mSmtp
←[2m2023-08-13T15:57:42.743961Z←[0m ←[34mDEBUG←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Listener shutting down. ←[3mevent←[0m←[2m=←[0m"shutdown" ←[3minstance←[0m←[2m=←[0m"imaptls" ←[3mprotocol←[0m←[2m=←[0mImap
←[2m2023-08-13T15:57:42.744007Z←[0m ←[34mDEBUG←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Listener shutting down. ←[3mevent←[0m←[2m=←[0m"shutdown" ←[3minstance←[0m←[2m=←[0m"jmap" ←[3mprotocol←[0m←[2m=←[0mJmap
←[2m2023-08-13T15:57:42.744191Z←[0m ←[34mDEBUG←[0m ←[2mutils::listener::listen←[0m←[2m:←[0m Listener shutting down. ←[3mevent←[0m←[2m=←[0m"shutdown" ←[3minstance←[0m←[2m=←[0m"submissions" ←[3mprotocol←[0m←[2m=←[0mSmtp

(When redirecting the output to an txt file, ← is an ASCII ESC byte \x1B.)

Im currently evaluating stalwart-mail for my livingroom-server which runs Windows. The log is hardly readable with this escape sequences. The escape sequences even appear when redirecting output into a txt file.

My current plan is to use the https://github.com/winsw/winsw/releases service wrapper. This wrapper logs stdout (and stderr) into a file. So I want to use the stdout logging of stalwart-mail to reduce the number of logfiles.

Could an option be added to disable the escape sequences?
If the output is not written to a console (= redirected to a pipe), escape sequences should be disabled automatically IMO.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:
Configuration:

[global.tracing]
method = "stdout"
level = "trace"

In cmd.exe: stalwart-mail.exe --config=config.toml

Version

v0.3.1

What database are you using?

None

What blob storage are you using?

Local

Where is your directory located?

None

What operating system are you using?

Windows

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

An android device is causing panics when fetching e-mail via IMAP.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

1. start stalwart-mail
2. fetch emails over IMAP
   The panic only occurs with a specific device (Galaxy Tab S3, see logs), on another android device the error does not happen.

Version

v0.3.x

Tested with v0.3.5 and v0.3.6 (but the messages differ).

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

None (Memory)

What operating system are you using?

Linux

Relevant log output

2023-08-30T17:47:25.475046Z  INFO session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: utils::listener::listen: context="tls" event="handshake" version=TLSv1_2 cipher=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
2023-08-30T17:47:25.520172Z TRACE session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: imap::core::client: event="read" data="437 CAPABILITY\r\n" size=16
2023-08-30T17:47:25.546455Z TRACE session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: imap::core::client: event="read" data="438 ID (\"name\" \"com.samsung.android.email.provider\" \"os\" \"android\" \"os-version\" \"9; PPR1.180610.011\" \"vendor\" \"samsung\" \"x-android-device-model\" \"SM-T820\" \"AGUID\" \"SaAlOCuK5s/fta0VjnMkzV/qHHk=\")\r\n" size=196
2023-08-30T17:47:25.570189Z TRACE session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: imap::core::client: event="read" data="439 LOGIN [email protected] \"xxxxxxxxxxxxxxxx\"\r\n" size=54
2023-08-30T17:47:26.330078Z TRACE session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: imap::core::client: event="read" data="440 NAMESPACE\r\n" size=15
2023-08-30T17:47:26.370820Z TRACE session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: imap::core::client: event="read" data="441 ENABLE CONDSTORE\r\n" size=22
2023-08-30T17:47:26.396834Z TRACE session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: imap::core::client: event="read" data="442 ENABLE QRESYNC\r\n" size=20
2023-08-30T17:47:26.422859Z TRACE session{instance="imap" protocol=Imap remote.ip="::xxxx:xxx.xxx.xxx.xxx" remote.port=xxxx}: imap::core::client: event="read" data="443 SELECT \"INBOX\" (QRESYNC (1693237464 16582))\r\n" size=49
thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', crates/imap/src/op/select.rs:96:86
stack backtrace:
   0:     0x55c61847248a - std::backtrace_rs::backtrace::libunwind::trace::he648b5c8dd376705
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55c61847248a - std::backtrace_rs::backtrace::trace_unsynchronized::h5da3e203eef39e9f
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55c61847248a - std::sys_common::backtrace::_print_fmt::h8d28d3f20588ae4c
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:65:5
   3:     0x55c61847248a - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hd9a5b0c9c6b058c0
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55c61754a41f - core::fmt::rt::Argument::fmt::h0afc04119f252b53
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/fmt/rt.rs:138:9
   5:     0x55c61754a41f - core::fmt::write::h50b1b3e73851a6fe
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/fmt/mod.rs:1094:21
   6:     0x55c61843bf86 - std::io::Write::write_fmt::h184eaf275e4484f0
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/io/mod.rs:1714:15
   7:     0x55c6184739ff - std::sys_common::backtrace::_print::hf58c3a5a25090e71
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x55c6184739ff - std::sys_common::backtrace::print::hb9cf0a7c7f077819
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x55c6184735a4 - std::panicking::default_hook::{{closure}}::h066adb2e3f3e2c07
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:269:22
  10:     0x55c6184745d7 - std::panicking::default_hook::h277fa2776900ff14
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:288:9
  11:     0x55c6184745d7 - std::panicking::rust_panic_with_hook::hceaf38da6d9db792
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:705:13
  12:     0x55c618474083 - std::panicking::begin_panic_handler::{{closure}}::h2bce3ed2516af7df
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:595:13
  13:     0x55c618474016 - std::sys_common::backtrace::__rust_end_short_backtrace::h090f3faf8f98a395
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys_common/backtrace.rs:151:18
  14:     0x55c618474001 - rust_begin_unwind
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/panicking.rs:593:5
  15:     0x55c6172cb992 - core::panicking::panic_fmt::h4ec8274704d163a3
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:67:14
  16:     0x55c6172cbb62 - core::panicking::panic::hee69a8315e4031d6
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/core/src/panicking.rs:117:5
  17:     0x55c6177fc1d9 - core::option::Option<T>::unwrap_or_else::he3353b2765cf0af3
  18:     0x55c6177e2f46 - imap::core::client::<impl imap::core::Session<T>>::ingest::{{closure}}::hd1294962bc7f1b2b
  19:     0x55c6176d2594 - imap::core::session::<impl imap::core::Session<tokio_rustls::server::TlsStream<tokio::net::tcp::stream::TcpStream>>>::handle_conn::{{closure}}::h9b7d84348e5bc83f
  20:     0x55c6176cf169 - imap::core::session::<impl utils::listener::SessionManager for imap::core::ImapSessionManager>::spawn::{{closure}}::ha939d122c0d7ed6d
  21:     0x55c6176ccd7c - tokio::runtime::task::raw::poll::h4c7e79ece89738fc
  22:     0x55c6184d93d9 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::h5283d2d722d28cf4
  23:     0x55c6184e09b4 - tokio::runtime::task::raw::poll::ha9735b86bc80b728
  24:     0x55c6184c53c7 - std::sys_common::backtrace::__rust_begin_short_backtrace::h8061f8434d606e05
  25:     0x55c6184c50c0 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hda173c06dfcdc174
  26:     0x55c618476825 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc0b1022758ecac73
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/alloc/src/boxed.rs:1993:9
  27:     0x55c618476825 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h0c9654ebe7ad657e
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/alloc/src/boxed.rs:1993:9
  28:     0x55c618476825 - std::sys::unix::thread::Thread::new::thread_start::h04c8e9c7d83d3bd5
                               at /rustc/5680fa18feaa87f3ff04063800aec256c3d4b4be/library/std/src/sys/unix/thread.rs:108:17
  29:     0x7f3c9b990afa - start_thread
                               at ./nptl/pthread_create.c:442:8
  30:     0x7f3c9ba1173c - clone3
                               at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
  31:                0x0 - <unknown>

Code of Conduct

• I agree to follow this project's Code of Conduct

Which feature or improvement would you like to request?

I'd like to disable colored log messages.

Is your feature request related to a problem?

Currently, colored messages are always active for either logging to stdout and file.

ESC[2m2023-09-16T06:41:35.555227ZESC[0m ESC[32m INFOESC[0m ESC[2mutilsESC[0mESC[2m:ESC[0m Starting Stalwart Mail Server v0.3.7...

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

When you import the messages from a maildir the date reported on the mail is the time of the import.

This messes with the timeline of all emails.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

• Use a maildir with older messages
• import them

Version

v0.3.1

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

LDAP

What operating system are you using?

Linux

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

Which feature or improvement would you like to request?

SurrealDB can be used for both authentication purposes (as it supports auth out of the box) and as a DB backend, down the line it may also be possible to use Surreal as an anti-spam solution, depending on the storage strategy, the team at Surreal are working on ML fields (embedded into the DB), which allows Surreal to learn and compute potential spam based on multiple factors.

Is your feature request related to a problem?

Currrently only SQLite (which isn't scalable) and FoundationDB is supported (Surreal's also multi model).

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

I set up rspamd per the docs and sent a test email through Gmail. rspamd classified the message as spam due to:

MISSING_MID (2.5)
MISSING_FROM (2)
MISSING_TO (2)
MISSING_DATE (1)
MISSING_SUBJECT (0.5)

But when I view the message body through IMAP, all these headers are present. When I copy-paste the entire message into rspamd's Scan interface, none of these checks are failed.

How can we reproduce the problem?

Set up rspamd per the docs, send a test email

Version

v0.3.1

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

SQLite

What operating system are you using?

Linux

Relevant log output

2023-08-06T11:29:10.659668Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::ehlo: context="dnsbl" event="invalid-reply" query="51.218.85.209.zen.spamhaus.org
." reply=[127.255.255.254]                                                                                                                                                                                            
2023-08-06T11:29:10.882833Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::ehlo: context="dnsbl" event="invalid-reply" query="mail-ej1-f51.google.com.dbl.sp
amhaus.org." reply=[127.255.255.254]                                                                                                                                                                                  
2023-08-06T11:29:10.899384Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::ehlo: context="spf" event="lookup" identity="ehlo" domain="mail-ej1-f51.google.co
m" result=None                                                                                                                                                                                                        
2023-08-06T11:29:10.899418Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::ehlo: context="ehlo" event="ehlo" domain="mail-ej1-f51.google.com"               
2023-08-06T11:29:10.945022Z  INFO session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: utils::listener::listen: context="tls" event="handshake" version=TLSv1_3 cipher=TLS13_AES_256_GC
M_SHA384                                                                                                                                                                                                              
2023-08-06T11:29:11.036322Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::mail: context="iprev" event="lookup" result=pass ptr="mail-ej1-f51.google.com."  
2023-08-06T11:29:11.037158Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::ehlo: context="dnsbl" event="invalid-reply" query="mail-ej1-f51.google.com.dbl.sp
amhaus.org." reply=[127.255.255.254]                                                                                                                                                                                  
2023-08-06T11:29:11.085300Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::mail: context="spf" event="lookup" identity="mail-from" domain="mail-ej1-f51.goog
le.com" sender="<redacted>" result=Pass                                                                                                                                                                     
2023-08-06T11:29:11.085343Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::mail: context="mail-from" event="success" address="<redacted>"         
2023-08-06T11:29:11.088553Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::rcpt: context="rcpt" event="success" address="<redacted>"                       
2023-08-06T11:29:11.129984Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::data: context="dkim" event="verify" return_path="<redacted>" from="bobo
[email protected]" result=["pass"]                                                                                                                                                                                     
2023-08-06T11:29:11.130395Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::data: context="arc" event="verify" return_path="<redacted>" from="bobob
[email protected]" result=none                                                                                                                                                                                          
2023-08-06T11:29:11.145109Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::data: context="dmarc" event="verify" return_path="<redacted>" from="bob
[email protected]" dkim_result=pass spf_result=pass                                                                                                                                                                   
2023-08-06T11:29:13.756379Z DEBUG session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::inbound::data: Milter filter(s) accepted message. context="milter" event="accept" modifica
tions="CHANGE_HEADER (index: 1, X-Spam-Status: Yes)"                                                                                                                                                                  
2023-08-06T11:29:13.758009Z  INFO session{instance="smtp" protocol=Smtp remote.ip="209.85.218.51" remote.port=62605}: smtp::queue::spool: Message queued for delivery. context="queue" event="scheduled" id=744636553 
from="<redacted>" nrcpts=1 size=4959                                                                                                                                                                        
2023-08-06T11:29:13.793373Z  INFO delivery{id=744636553 return_path="<redacted>" nrcpt=1 size=4959}:attempt{domain="mydomain" attempt_number=0}: smtp::outbound::local: context="deliver_local" event="delivere
d" rcpt="<redacted>"                                                                                                                                                                                                 
2023-08-06T11:29:13.793495Z  INFO delivery{id=744636553 return_path="<redacted>" nrcpt=1 size=4959}: smtp::outbound::delivery: Delivery completed. context="queue" event="completed"

Code of Conduct

• I agree to follow this project's Code of Conduct

Which feature or improvement would you like to request?

I'd like to see this feature: That by-default Stalwart would generate a configuration for DKIM double-signing with both RSA and Ed25519.

Increased adoption of newer EC DKIM algorithms (such as Ed25519) would encourage other software to add support for it. The very least it would be an useful test-case and an example of a (popular) MTA that supports EC DKIM.

In the end this would help reduce ossification of the email ecosystem with basically no cost.

Is your feature request related to a problem?

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

Hi!

First of all I wanted to tell you that's this piece of software looks great!

I would like to try it but for now my deal-breaker is lack of documentation for spam filtering - spamassasin / rspamd (I'd focus on the latter one)

Could you add some examples of configuration? In the configs there only one mention of spamc

What happened?

The documentation on ARC sealing states:

The list of signatures to use for sealing is configured by the auth.arc.sign parameter, for example:

[auth.arc]
seal = ["rsa"]

There's a small discrepancy there between sign and seal. I'm assuming it should be seal, as that is in the config generated by stalwart-install.

The example from the docs seems to work fine with a single value. However, when I add an additional value, e.g. seal = ["mail1", "mail2",], an error is thrown upon startup of the stalwartlabs/mail-server:v0.3.3 image:

Invalid configuration file: Property "auth.arc.seal.0" cannot have multiple values.

How can we reproduce the problem?

1. Generate a config using stalwart-install;
2. Modify the config:
   1. Add an extra [signature."sig2"] block;
   2. Add the name of the second signature to the auth.arc.seal list;
3. Start the mail-server image using the config;
4. See how it immediately crashes.

Version

v0.3.3 (Note: the bug template only goes up to 0.3.1)

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

SQLite

What operating system are you using?

Docker

Relevant log output

Invalid configuration file: Property "auth.arc.seal.0" cannot have multiple values.

Code of Conduct

• I agree to follow this project's Code of Conduct

Which feature or improvement would you like to request?

I'd like to see this feature:
implement dynamic values for the file_path signing section and more

The desired result:
[signature."some key"]
private-key = [ { if = "sender-domain", in-list = "sql/domains", then = "/opt/stalwart-mail/etc/dkim/${0}.key" },{ else = [] } ]
domain = [ { if = "sender-domain", in-list = "sql/domains", then = "${0}" },{ else = [] } ]
selector = "stalwart"
headers = ["From", "To", "Date", "Subject", "Message-ID"]
algorithm = "rsa-sha-256"
canonicalization = "relaxed/relaxed"
set-body-length = false
report = true

Is your feature request related to a problem?

Currently is not possible to automate [signature.".."] section.
In the situation when you have more than 20 domains or even 5 domains maintaining dkim keys manually by changing files is expensive and insecure because you need to restart the server and physically change the config whenever you need to regenerate dkim or add a new domain.

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

I tried to transfer my old e-mail account to Stalwart via IMAP. After only a few thousand e-mails I got the message "Disk quota exceeded.". The account had a quota of 100 GB and I got this error having transferred only 1-2 GB to it. I changed the quota for the account to 0 and it worked. The complete data directory is only 8 GB after the finished transfer of all my e-mails (including other accounts). So somewhere there must be an error in the quota calculation.

How can we reproduce the problem?

Transfer big amounts of e-mails into an account via IMAP with a generous quota with a setup like described below

Version

v0.3.x

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

SQLite

What operating system are you using?

Linux

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

The docs regarding passwords list Argon2 as an algorithm, as the first option, and described as "the winner of the Password Hashing Competition", so someone looking into how to set up their directory might assume that Argon2 is a good default option.

Stalwart doesn't seem to have any tool to hash passwords so I installed my distro's argon2 package, which provides a (the?) argon2 command line tool.

It appears that there are several Argon2 variants: Argon2i, Argon2d and Argon2id. The argon2 command defaults to Argon2i.

Stalwart checks for Argon2 hashes but unconditionally uses Argon2::default() from the Argon2 crate, which defaults to Argon2id.

I think instead, Stalwart should use PasswordHash::Parse and use the resulting PasswordHash object to verify the password.

How can we reproduce the problem?

Use the argon2 command to hash a password and put the result in the database. Try to authenticate.

Authentication fails, claiming the hash is invalid.

Version

v0.3.1

What database are you using?

None

What blob storage are you using?

None

Where is your directory located?

SQLite

What operating system are you using?

Linux

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

The management documentation references this command but the Docker containers seem not to have it, at least the mail-server container.

I cannot setup ED25519 keypair for DKIM signing.

To create the keypair I used this commands:

openssl genpkey -algorithm ed25519 -out ed_private.key
openssl pkey -in ed_private.key -pubout -out ed_public.key

This is my config

[signature."ed25519"]
public-key = "MCowBQYDK2VwAyEAlzr6RDO6iDZHCeSaP0KBy4vq/KYvKB9bPEpWpNhFHYY="
private-key = "file:///opt/stalwart-mail/etc/dkim/ed_private.key"
domain = "jotone.eu"
selector = "stalwart_ed"
headers = ["From", "To", "Date", "Subject", "Message-ID"]
algorithm = "ed25519-sha256"
canonicalization = "simple/simple"
set-body-length = true
report = true

And this is the error that the server reports:

INFO utils: Starting Stalwart Mail Server v0.3.0...
Invalid configuration file: Failed to base64 decode private key for signature.ed25519.private-key.

While looking at the code, it seems subaddressing is based on the scheme "[email protected]". This reveals the actual account and is known by spammer so they drops anything after the "+" sign anyway.

Would it be possible to use a regex here to split the string instead ?

My actual email server allows to use a regex here, so it can behave like gmail is doing (ignoring any dots in the user name ("a.b" is similar to "ab") or like I prefer, using dot as a separator ("[email protected]") with spammer changing for each site I'm registering upon.

The configuration would be a pair:

1. Regex to match against (if it matches, the subaddressing is accepted)
2. Email account to use instead

So instead of "[email protected]", the configuration would be ("/jane\+([^@]+)@domain.org", "[email protected]")
This could allow a much more useful feature like ("/spam\.([^@]+)@domain.org", "[email protected]") that's not revealing the underlying account name.

Hello!
First of all, congrats on the hard work, a product like this is very needed.
I have a question in respect to the docs: is there some place (that I'm missing) which describes how to create a new user?
Yes, I've read the page. But it's a little bit difficult for me to grasp the details. By example, the sample schema is given for sqlite, if you try with Postgres schema creation will fail successfully, because of boolean default 1. And yes, I can go with trial and error, by making it boolean default true or integer default 1, but I think user management is a core activity that deserves a little bit more documentation.
The same goes for the encryption of the password. One can argue "you can look in the code", and that's a valid point too, but still I think this item deserves a couple of examples of possible way in handling hashing (by example Postgres has sha256 as a function that can be easily applied, but do I need to append the qualifier in front or not?). Again, tinkering and trial and error is an option, but I think it would be easier for your user base if that would be documented the way you intended it to be used.
Thank you!

This is not an issue when using stalwart-cli to import/export on a local system. When trying to export a large mailbox over the internet, for me one of the blob downloads somewhat reliably fails after something like 5000 to 15000 requests. An example of an error is:

Failed to download blob: Transport error: error sending request for url (): operation timed out

This fails the entire procedure. I think I had previously had a similar issue, but the solution is to just repeat failed requests a few times before giving up completely. This particular issue happened to me while exporting, but I assume importing is also affected.

Which feature or improvement would you like to request?

Dear @stalwartlabs team,

Can you add supports of :

• SCRAM-SHA-1
• SCRAM-SHA-1-PLUS
• SCRAM-SHA-256
• SCRAM-SHA-256-PLUS
• SCRAM-SHA-512
• SCRAM-SHA-512-PLUS
• SCRAM-SHA3-512
• SCRAM-SHA3-512-PLUS

You can add too:

• SCRAM-SHA-224
• SCRAM-SHA-224-PLUS
• SCRAM-SHA-384
• SCRAM-SHA-384-PLUS

A "big" list has been done in last link of this ticket.

SCRAM-SHA-1(-PLUS):

• https://tools.ietf.org/html/rfc5802
• https://tools.ietf.org/html/rfc6120

SCRAM-SHA-256(-PLUS):

• https://tools.ietf.org/html/rfc7677 since 2015-11-02
• https://tools.ietf.org/html/rfc8600 since 2019-06-21: https://mailarchive.ietf.org/arch/msg/ietf-announce/suJMmeMhuAOmGn_PJYgX5Vm8lNA

SCRAM-SHA-512(-PLUS):

• https://tools.ietf.org/html/draft-melnikov-scram-sha-512

SCRAM-SHA3-512(-PLUS):

• https://tools.ietf.org/html/draft-melnikov-scram-sha3-512

SCRAM BIS: Salted Challenge Response Authentication Mechanism (SCRAM) SASL and GSS-API Mechanisms:

• https://tools.ietf.org/html/draft-melnikov-scram-bis

-PLUS variants:

• RFC5056: On the Use of Channel Bindings to Secure Channels: https://tools.ietf.org/html/rfc5056
• RFC5929: Channel Bindings for TLS: https://tools.ietf.org/html/rfc5929
• Channel-Binding Types: https://www.iana.org/assignments/channel-binding-types/channel-binding-types.xhtml
• RFC9266: Channel Bindings for TLS 1.3: https://tools.ietf.org/html/rfc9266

IMAP:

• RFC9051: Internet Message Access Protocol (IMAP) - Version 4rev2: https://tools.ietf.org/html/rfc9051

LDAP:

• RFC5803: Lightweight Directory Access Protocol (LDAP) Schema for Storing Salted: Challenge Response Authentication Mechanism (SCRAM) Secrets: https://tools.ietf.org/html/rfc5803

HTTP:

• RFC7804: Salted Challenge Response HTTP Authentication Mechanism: https://tools.ietf.org/html/rfc7804

JMAP:

• RFC8621: The JSON Meta Application Protocol (JMAP) for Mail: https://tools.ietf.org/html/rfc8621

2FA:

• Extensions to Salted Challenge Response (SCRAM) for 2 factor authentication: https://tools.ietf.org/html/draft-melnikov-scram-2fa

IANA:

• Simple Authentication and Security Layer (SASL) Mechanisms: https://www.iana.org/assignments/sasl-mechanisms/sasl-mechanisms.xhtml

Linked to:

• scram-sasl/info#1
• #176

Is your feature request related to a problem?

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

I can't receive any new emails after importing a maildir exported using mbsync from a https://maddy.email server

How can we reproduce the problem?

I exported my emails from maddy using mbsync with this config:

CopyArrivalDate yes
Sync Pull
Create Near
Remove Near
Expunge Near

IMAPAccount maddy
Host uwu.kkx.one
User [email protected]
Pass test69420
SSLType IMAPS

IMAPStore maddy-remote
Account maddy

MaildirStore maddy-local
Path ~/mail/test/
Inbox ~/mail/test/INBOX
SubFolders Verbatim

Channel maddy
Far :maddy-remote:
Near :maddy-local:
Patterns *
SyncState *

This is the exported maildir: test.zip

Then I imported this into an empty account using stalwart-cli -u https://jmap.kkx.one import messages --format maildir-nested test ~/mail/test.

This created a bunch of conflicting mailboxes, like "Junk" and "Junk Mail", though I wouldn't really expect this to break something.

Then I tried sending myself an email both from GMail and ProtonMail, none of them appeared in my inbox either using Thunderbird or K-9 Mail.

When I did this with my main account and more emails, Thunderbird seemed to work just fine, while K-9 Mail didn't see any new emails.

This feels like an issue with either mailbox IDs or message IDs, mbsync did complain that UIDVALIDITY was missing on every folder, and it created a .uidvalidity file in every mailbox.

Version

v0.3.x

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

PostgreSQL

What operating system are you using?

Docker

Relevant log output

Didn't manage to see anything relevant amidst all the bot login attempts spam in the logs, other than the messages I tried sending myself being "delivered successfully".

Code of Conduct

• I agree to follow this project's Code of Conduct

Discussed in #40

Stalwart Mail Server is a relatively new product, being only 2 years old. The primary focus during its development has been on ensuring a robust set of features to make it competitive in the market. Admittedly, performance tuning hasn't been the top priority until now.

On Q1 2024, the primary objective will be to focus on performance enhancements. The goal is not just to match but hopefully to surpass the performance metrics of other established mail servers.

Which feature or improvement would you like to request?

I'd like to see TLS ciphersuite functionality a bit more robust against (mis)configurations.

• Cipher suites that aren't supported shouldn't always cause a fatal error. This makes it really difficult to figure out what ciphers are supported (or which primitive is lacking) as there's no HIGH:MEDIUM:!LOW shorthand (like OpenSSL has). This will cause really annoying issues should any of the ciphersuites be deprecated in the future and not included in the builds.

• Too restrictive of a cipher set should be warned against. E.g. only two "strong" TLSv1.2 ciphersuites is really not a good idea. Ideally it would also fall back to a safe default if there are none valid for a specific protocol. Security-related misconfigurations should in general be hard.

• The example configuration should also include ciphers needed for elliptic certificates. So TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 in addition to current TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256. This omission can cause misconfigurations.

• Support of secure (PFS) ciphers such as TLS_ECDHE_ECDSA_WITH_AES_256_CCM, TLS_ECDHE_ECDSA_WITH_AES_128_CCM (and their RSA equivalents) should be added to the Docker images.

• Support for TLSv1.1 in the Docker images.

I will note that I would certainly love to run a TLSv1.3-only MTA, but considering how the fallback is usually going to be plaintext, TLSv1.1 and/or "sufficient" ciphersuites are infinitely more preferable.

Code of Conduct

• I agree to follow this project's Code of Conduct

First off, I'm using the docker image stalwartlabs/mail-server:latest on arm64.

Tried connecting to OpenLDAP via ports 389 (starttls) and 636 (ssl) but it ends up timing out with the following:

2023-07-18T08:42:12.954703Z  WARN directory: Directory timed out context="directory" event="error" protocol="ldap"
2023-07-18T08:42:23.184702Z  WARN directory: LDAP directory error context="directory" event="error" protocol="ldap" reason=I/O error: invalid peer certificate: UnknownIssuer

Checking the logs from OpenLDAP I noticed the following:

64b64e8b.0e4024ae 0xffff436ea8e0 conn=6081 fd=18 ACCEPT from IP=[__cleared__]:38747 (IP=[::]:636)
64b64e8b.10425005 0xffff3c7238e0 TLS: can't accept: (unknown).
64b64e8b.104428ff 0xffff3c7238e0 conn=6081 fd=18 closed (TLS negotiation failure)
64b64ead.127b6aeb 0xffff3aa1e8e0 conn=6082 fd=18 ACCEPT from IP=[__cleared__]:50649 (IP=[::]:636)
64b64ead.149bd09f 0xffff411518e0 TLS: can't accept: (unknown).
64b64ead.149d4c28 0xffff411518e0 conn=6082 fd=18 closed (TLS negotiation failure)
64b64ece.16a29781 0xffff421578e0 conn=6083 fd=18 ACCEPT from IP=[__cleared__]:40531 (IP=[::]:636)
64b64ece.16a76b25 0xffff421578e0 TLS: can't accept: error:0A0000C1:lib(20)::reason(193).
64b64ece.16a8d0f6 0xffff421578e0 conn=6083 fd=18 closed (TLS negotiation failure)
64b64eef.1ada6dc0 0xffff2aa1e8e0 conn=6084 fd=18 ACCEPT from IP=[__cleared__]:38585 (IP=[::]:636)
64b64eef.1cee0b64 0xffff436ea8e0 TLS: can't accept: (unknown).
64b64eef.1cefd00d 0xffff436ea8e0 conn=6084 fd=18 closed (TLS negotiation failure)

I'm using Let's Encrypt certs, they're currently valid, and I'm not getting any errors from any other services. I'm guessing there is an issue with ca certs not being present in the docker image or they're not getting imported by rustls.

I dug around the source and found where/how to disable certificate validation, which did allow me to connect to my directory, but I also noticed that where to put these options wasn't in the documentation:

[directory."ldap"]
type = "ldap"
address = "ldaps://ldap.xyz.com"
allow-invalid-certs = true

Thanks!

What happened?

I get this panic with stacktrace:

thread 'tokio-runtime-worker' panicked at 'attempt to calculate the remainder with a divisor of zero', crates\smtp\src\reporting\scheduler.rs:209:13
stack backtrace:
   0:     0x7ff77b9c18c1 - <unknown>
   1:     0x7ff77a74b8eb - <unknown>
   2:     0x7ff77b9a4829 - <unknown>
   3:     0x7ff77b9c3adb - <unknown>
   4:     0x7ff77b9c3899 - <unknown>
   5:     0x7ff77b9c4667 - <unknown>
   6:     0x7ff77b9c40d3 - <unknown>
   7:     0x7ff77b9c4059 - <unknown>
   8:     0x7ff77b9c4044 - <unknown>
   9:     0x7ff77bdb6785 - <unknown>
  10:     0x7ff77bdb6952 - <unknown>
  11:     0x7ff77b80d852 - <unknown>
  12:     0x7ff77b808249 - <unknown>
  13:     0x7ff77b7fb1c0 - <unknown>
  14:     0x7ff77b7f8bb9 - <unknown>
  15:     0x7ff77ba42837 - <unknown>
  16:     0x7ff77ba4d230 - <unknown>
  17:     0x7ff77ba2885d - <unknown>
  18:     0x7ff77ba2834a - <unknown>
  19:     0x7ff77b9bdbdb - <unknown>
  20:     0x7ff90eca7614 - BaseThreadInitThunk
  21:     0x7ff90fe426b1 - RtlUserThreadStart

How can we reproduce the problem?

I'm just letting run stalwart-mail v0.3.5 on a Windows 10 22H2 machine. It runs wrapped by WinSW under the account "Local Service".

The panic shows up repeatedly in the stderr log. I configured hash = 0

Version

v0.3.5

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

Memory

What operating system are you using?

Windows

Relevant log output

Code of Conduct

• I agree to follow this project's Code of Conduct

I would like to be able to output to a file + open-telemetry. So it would be nice to be able to have multiple logging outputs.

The stalwart-cli administration tool requires passing the server URL and
credentials either as CLI arguments or in an interactive prompt.

The former is rather cumbersome when having to run a lot of commands, and can
cause passwords to leak in htop for example.

The latter is also inconvenient and not possible in automated shell scripts.

Would it be possible to be able to set those parameters through some
environment variables as well?

What happened?

2m2023-07-30T12:27:09.783319Z�[0m �[33m WARN�[0m �[2mdirectory�[0m�[2m:�[0m SQL directory error �[3mcontext�[0m�[2m=�[0m"directory" �[3mevent�[0m�[2m=�[0m"error" �[3mprotocol�[0m�[2m=�[0m"sql" �[3mreason�[0m�[2m=�[0merror returned from database: syntax error at or near "AND"

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

Version

v0.3.1

What database are you using?

None

What blob storage are you using?

Local

Where is your directory located?

PostgreSQL

What operating system are you using?

Docker

Relevant log output

2m2023-07-30T12:27:09.783319Z�[0m �[33m WARN�[0m �[2mdirectory�[0m�[2m:�[0m SQL directory error �[3mcontext�[0m�[2m=�[0m"directory" �[3mevent�[0m�[2m=�[0m"error" �[3mprotocol�[0m�[2m=�[0m"sql" �[3mreason�[0m�[2m=�[0merror returned from database: syntax error at or near "AND"

Code of Conduct

• I agree to follow this project's Code of Conduct

Years ago, I started writing my own SMTP server and IMAP Server, and years later, when I became a better programmer, I started to redo it, but never finished because I recently became proficient with Rust and wanted to re-write it again. So this project is really exciting for me. Postfix / Dovecot has been the FOSS go-to email setup for far too long!

One of the things I had planned for my Rust re-write was to abstract away the SASL authentication, much like how Postfix can defer to Dovecot's SASL service for authentication. In my opinion, SASL authentication should be handled by a different application, because it would probably be desirable to have a multitude of backing identity stores (LDAP servers, Kerberos servers, SQL databases, etc.), and the complexity of this alone can grow so large as to make it not worth it to incorporate into the main SMTP / IMAP server. The configuration for a SMTP + IMAP + SASL server could be enormous! For that matter, if SASL is defined as a protocol, different SASL servers could be swapped out for each other as needed.

More concretely, my plans for implementing this were to use Google's Protocol Buffers to define a dead-simple SASL protocol. Using Protocol Buffers means that anybody could pretty easily create a compatible SASL server in just about any language and not even have to detail with the wire protocol.

I haven't drafted such a protocol formally, but off the top of my head, I think it would have just two principal operations:

1. Assert one SASL message from the client, and receive either an authentication continuation, success, or failure response, as well as the authzid.
2. Retrieve the supported SASL mechanisms (which might be dependent upon the backing identity stores configured), which can be used to populate the SASL mechanisms returned by the EHLO command.

I know the other Stalwart mail server apps use some sort of configurable scripting query against a database for looking up users, but would this be something you're interested in? I would be willing to work on this and create such a SASL server.

I'd qualify this by saying that support for a SASL server does not preclude simpler methods, such as a simple passwd file or something; in fact, I would recommend that so that users do not have to set up a SASL server.

Anyway, thanks for working on this cool project!

I'm currently running the official docker arm64 container, trying to get Stalwart to work with an OpenLDAP directory that has passwords hashed with argon2.

Trying to auth against some of these accounts, I get the following error logged from Stalwart:
WARN directory::secret: Unsupported password hash algorithm context="directory" event="error" algorithm="ARGON2"

I took a peek through the relevant source and it looks like it isn't able to evaluate/match the passwords to argon2 because OpenLDAP stores the passwords differently than expected by Stalwart. I see a condition that checks if the password starts with $argon2 but OpenLDAP generally stores it's passwords with the algorithm stored within brackets, followed by the password/hash after that.

An example of a password stored in OpenLDAP in this format would be the following:
{ARGON2}$argon2id$v=19$m=65536,t=2,p=1$xRd2axJow818R4IVEjqdbg$y3Mm4pHnPEh/5PgyU+SWRCj9M44vY45hejY3RSdE72Y

It also looks like there are some types of crypt that would fall into a similar pit, i.e.:

{CRYPT}$6$rounds=50000$4XSqWr3MK8MtGRs$cz6fsr6uqwU5tDLhrnZRCra.xmuOm.lsZKeybuytG3jTE8pwP45j29NHP/C15ejdRXp0ZIqppIH2vm4jYTy/S1

{CRYPT}$6$$ek/ucQg0IM8SQLyD2D66mpoW0vAF26eA0/pqoN95V.F0nZh1IFuENNo0OikacRkDBk5frNqziMYMdVVrQ0o.51

{CRYPT}$1$$IqH.8BV8l9Uhl3qbv9FMq/

{CRYPT}$5$$zuuTJtok6EzzyDhHOum0I1x.ucYLJxAGeQgHLQ/ohK0

Thanks!

Which feature or improvement would you like to request?

I'd like to see: Stallwart uses mailinglists for community communication.

Is your feature request related to a problem?

When building a mailserver, use that mailserver.

Code of Conduct

• I agree to follow this project's Code of Conduct

The Stalwart mail service refuses to start with the following error,
even when no listener of the jmap kind is configured (IMAP-only setup).

thread 'main' panicked at 'Invalid configuration file:
Missing property "jmap.directory".',
/build/source/crates/utils/src/lib.rs:68:17

What happened?

When you mail to a TLS capable server but it does not have a matching TLS setup it now gives an error:

<[[email protected]](mailto:[email protected])> (TLS error from 'appmaildev.com': I/O error: tls handshake eof)

I have set TLS optional:

[queue.outbound.tls]
starttls = "optional"

Although the documentation is correct that it should do TLS when the server offers the option, but when set to optional you say when possible. If the TLS does not match you could say we can't and should fallback?

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

• Create a test email here: https://appmaildev.com/en/dkim
• Send email

Version

v0.3.5

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

LDAP

What operating system are you using?

Linux

Relevant log output

attempt{domain="appmaildev.com" attempt_number=1}: smtp::outbound::delivery: context="tls" event="failed" mx="appmaildev.com" error=I/O error: tls handshake eof

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

Which feature or improvement would you like to request?

I tried to configure stalwart with lldap as the LDAP directory backend, and after a lot of debugging I found out lldap doesn't provide the userPassword field¹.
lldap uses OPAQUE as the password authenticatio², is this a feature stalwart could implement?
I'd be happy to look into it but my LDAP knowledge at the moment is limited.

Is your feature request related to a problem?

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

hey, thank you for making this!

Just so you know, there are efforts to package this for release in nixos
NixOS/nixpkgs#234672

The PR packages the binaries, the next step will be to make a systemd service. If you ever have relevant files, feel free to reference them here.
If we get to making a systemd service before, I would be happy to tag you if you are interested in what we come up with.

(otherwise this is mostly for information that your work is very much appreciated!).

AUTH is not advertised when connected to SMTP without TLS,
even when some session.auth.mechanisms are set.
(Due to the condition !stream.is_tls() in src/inbound/ehlo.rs:173).

There is no way to lift this restriction for test setups or when within a
trusted network (VPN).

It would be nice to have an option allowing this, similar to the existing
imap.auth.allow-plain-text for IMAP.

What happened?

After configuring the stalwartlabs/smtp-server:v0.3.3 image, it crashes with "file not found", because there's no stalwart-mail binary in /usr/local/bin. I'm guessing it wants stalwart-smtp instead.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

$ podman run --rm -it --entrypoint=/bin/sh docker.io/stalwartlabs/smtp-server:v0.3.3
# ls /usr/local/bin
configure.sh  entrypoint.sh  stalwart-cli  stalwart-install  stalwart-smtp

Version

v0.3.1

What database are you using?

None

What blob storage are you using?

None

Where is your directory located?

LDAP

What operating system are you using?

Docker

Relevant log output

stalwart-smtp[33592]: /usr/local/bin/entrypoint.sh: 10: exec: /usr/local/bin/stalwart-mail: not found

Code of Conduct

• I agree to follow this project's Code of Conduct

I set up a MySQL server as a directory and when starting the mail server everything works for about 5 minutes, after that I get the following error message:

pool timed out while waiting for an open connection

Here is my pool config:

[directory."sql".pool]
max-connections = 10
min-connections = 0

When I set max-connections to something like 100 it takes about 30 minutes for the error to show up so I'm thinking it might have something to do with connections silently failing until all connections in the pool have failed.

What happened?

With the basic authentication it is unclear which user is used for ldap authentication or can you set a admin user globally? https://stalw.art/docs/management/overview#basic

How can we reproduce the problem?

Configure with ldap as backend and how do you become a admin?

Version

v0.3.1

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

LDAP

What operating system are you using?

Linux

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

See https://openid.net/specs/openid-connect-registration-1_0.html#RegistrationRequest

Very cool project! I am really excited to see and use the 1.0.0 release of the Stalwart mail servers. But why make all of them a single application?

What happened?

When specifying a custom sender in thunderbird, the email just goes through from any address and is not checked against the directory.

How can we reproduce the problem?

I can reproduce the problem by doing the following steps:

• Send an email with "From" value set to arbitrary address

Version

v0.3.x

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

PostgreSQL

What operating system are you using?

Docker

Relevant log output

2023-09-03T18:13:19.989365Z TRACE session{instance="submissions" protocol=Smtp remote.ip="::ffff:185.17.14.144" remote.port=59150}: smtp::inbound::session: event="read" data="MAIL FROM:<[email protected]> BODY=8BITMIME SIZE=346\r\n" size=57
2023-09-03T18:13:19.989414Z DEBUG session{instance="submissions" protocol=Smtp remote.ip="::ffff:185.17.14.144" remote.port=59150}: smtp::inbound::mail: context="mail-from" event="success" address="[email protected]"

Code of Conduct

• I agree to follow this project's Code of Conduct

Which feature or improvement would you like to request?

Are there plans to add support for the PROXY protocol? I'm mostly interested in the SMTP server, though it would be useful addition for IMAP as well. Since JMAP is HTTP based, proxies are probably already supported with headers.

Is your feature request related to a problem?

I'm evaluating mail-server on a (small) Kubernetes cluster and want to route all traffic through nginx (as ingress controller). This works fine, with one caveat: the mail server sees all incoming connections as coming from the ingress controller, instead of seeing the real client IP address. This makes the logging somewhat useless, breaks SPF verification (since the local IP address of my ingress controller is unlikely to be listed as a valid sender), breaks grey-listing, etc..

According to the docs of ingress-nginx on exposing TCP services, it's possible to use the PROXY protocol to pass the client IP address along with the connection. Of course, this is extra information that gets sent, so it would have to be implemented (and configured) by mail-server.

Code of Conduct

• I agree to follow this project's Code of Conduct

We are in the case that we can't send e-mail directly from our instance to the world. We have a relay which we use to send e-mails. I might have missed this documentation but is this possible?

Which feature or improvement would you like to request?

I would like to see the addition of fail-over support so that when a relay doesn't work, the emails can still be delivered using another relay.

Is your feature request related to a problem?

I cannot deliver emails using IPv4 (because of my cloud provider) so when IPv6 is available i configured the server to prefer it over IPv4. But when only IPv4 is available all the connections to port 25 are blocked

Discussion reference

#16

Code of Conduct

• I agree to follow this project's Code of Conduct

What happened?

I have a maildir format. When I try to import this I get weird items in the folder. When I switch to maildir-nested (it creates wrong folders) the e-mail is correct.

How can we reproduce the problem?

I have a mail setup with mailserver and I'm trying to import the messages.

Version

v0.3.1

What database are you using?

SQLite

What blob storage are you using?

Local

Where is your directory located?

LDAP

What operating system are you using?

Linux

Relevant log output

No response

Code of Conduct

• I agree to follow this project's Code of Conduct