Giter Club home page Giter Club logo

mistralmail's Introduction

MistralMail

MistralMail will be a production-ready, and easy to setup mail server. It consists of an SMTP server (both MSA and MTA) and an IMAP server all bundled in one executable (or just in one Docker image) with auto-generated TLS certificates.

⚠️ WIP: MistralMail is far from being production-ready! ⚠️

Usage

Setting up DNS records

MistralMail will not be able to generate TLS certificates without a correct DNS configuration. And of course you also won't be able to receive any emails. (But if you just want to configure it locally you can set TLS_DISABLE to true and skip this section.)

You need the following DNS records:

  • A record imap.yourdomain.com pointing to your MistralMail server ip.
  • A record mx.yourdomain.com pointing to your MistralMail server ip.
  • A record smtp.yourdomain.com pointing to your MistralMail server ip.
  • MX record point to mx.yourdomain.com.
  • SPF record pointing to your SMTP relay provider.

Running the MistralMail server

First you need to copy .env.sample to .env and configure all the needed environment variables.

When using HTTP challenge for TLS: make sure that ports 80 and 443 are opened for the automatic TLS certificate generation via Let's Encrypt.

Then you can run the Go main manually or with Docker.

Go:

source .env
go run cmd/mistralmail/*.go

Docker:

Everything needed is put into the docker-compose.yml file. If you don't want to build the image yourself you can use the prebuilt one present at denbeke/mistralmail.

docker-compose up mistralmail

Now you can create a user with the MistralMail CLI.

MistralMail exposes the following ports:

  • 25 for all incoming SMTP emails (MTA)
  • 587 for all outing SMTP emails (MSA)
  • 143 for IMAP
  • (443 & 80 for Let's Encrypt, when not using DNS challenge)
  • 9000 for the metrics
  • 8080 for the api & web server

Environment Variables

ENV Default value Description
HOSTNAME Hostname of the MistralMail mail server
SMTP_ADDRESS_INCOMING :25 Bind address for the listener of incoming email.
SMTP_ADDRESS_OUTGOING :587 Bind address for the listener of outgoing email.
IMAP_ADDRESS :143 Bind address for the listener of IMAP.
DATABASE_URL sqlite:test.db Database connection url.
Example using Postgres: postgresql://user:pass@localhost/mydatabase.
It defaults to a local Sqlite database.
SUBDOMAIN_INCOMING mx.{HOSTNAME} Domain for the incoming mail.
SUBDOMAIN_OUTGOING smtp.{HOSTNAME} Domain for the outgoing mail.
SUBDOMAIN_IMAP imap.{HOSTNAME} Domain for IMAP.
SMTP_OUTGOING_MODE RELAY Mode for delivering outgoing mail. Currently only RELAY mode is supported. So this means you have to configure an SMTP relay for sending out emails.
EXTERNAL_RELAY_HOSTNAME Hostname of the SMTP relay.
EXTERNAL_RELAY_PORT Port of the SMTP relay.
EXTERNAL_RELAY_USERNAME Username of the SMTP relay.
EXTERNAL_RELAY_PASSWORD Password of the SMTP relay.
EXTERNAL_RELAY_INSECURE_SKIP_VERIFY false Allow insecure connections to the SMTP relay.
TLS_DISABLE false Disable TLS for the MistralMail server.
TLS_ACME_CHALLENGE Type of the ACME challenge supports two types:
- HTTP: standard HTTP ACME challenge (need to open port 443 and 80 for this)
- DNS: challenge by DNS. Need to provide TLS_ACME_DNS_PROVIDER for this and configure the DNS provider API credentials.
TLS_ACME_EMAIL Email of the Let's Encrypt account.
TLS_ACME_ENDPOINT https://acme-v02.api.letsencrypt.org/directory Let's Encrypt endpoint. By default we use the production endpoint. If you want to test your configuration it is advised to test against staging to avoid rate limits: https://acme-staging-v02.api.letsencrypt.org/directory
TLS_ACME_DNS_PROVIDER DNS provider to be used for Let's Encrypt.
TLS_CERTIFICATES_DIRECTORY ./certificates Directory where TLS certificates are stored.
HTTP_ADDRESS :8080 Address of the webserver that serves the web interface and the API.
SECRET Encryption secret.
SENTRY_DSN Sentry DNS if you want to log errors to Sentry.
LOG_FULL_QUERIES false Log all queries with their parameters.
SPAM_CHECK_ENABLE false Enable the very basic spam check. Note that it sends all incoming messages to the Postmark Spam Check API.
METRICS_ADDRESS :9000 Prometheus metrics address.

Using the MistralMail Web UI

MistralMail comes with a basic web ui http://localhost:8080. At the moment it supports nothing more than basic user management and basic statistics.

mistralmail-web-ui

Using the MistralMail command line interface

You can use the MistralMail command line interface with Go or with Docker:

go run cmd/mistralmail-cli/*.go

or

docker-compose run mistralmail mistralmail-cli

Currently the CLI contains the following commands:

  • create-user: to create a new user.

  • reset-password to reset the password of a user.

Configuring your mail client

IMAP:

  • Server address: imap.yourdomain.com

  • Username: your email address

  • Port: 143

  • Security: STARTTLS

  • Authentication: password

SMTP:

  • Server address: smtp.yourdomain.com

  • Username: your email address

  • Port: 587

  • Security: STARTTLS

  • Authentication: password

Now you're all good to go!

Development

We use go work for updating files across multiple repo's:

go work init
go work use smtp
go work use imap-backend

Current state of MistralMail

SMTP server

The SMTP server is completely custom written and can be found here: mistralmail/smtp. It was written quite a while ago but it seems robust enough for now.

For outgoing emails we currently only support using an external relay like Mailgun or Sendgrid since we don't want to put too much time into debugging an MSA.

IMAP

For IMAP we wrote a SQL backend behind go-imap. It supports MySQL, Postgres and Sqlite. (Currently only Sqlite has actually been tested.)

This backend is very experimental and surely contains a lot of bug. The backend is also implemented in a very non-performant way. So don't expect that MistralMail will be able to handle large inboxes at its current state.

We dump the complete emails in the database at this moment. In the future we would like to add support for object storage for the actual mail bodies. But that's nothing for the near future.

Webmail

Currently there are no concrete plans to implement a webmail. But wouldn't it be nice to have it someday?

Web management

Instead of configuring everything via a CLI, it's also possible to use the very basic web ui. But this is still very basic.

SPAM

Another feature we are also not working on currently is anti-spam. Only SPF is checked at the moment. But nothing else.
A very basic X-Spam-Score header can be enabled by setting SPAM_CHECK_ENABLE to true. It is disabled by default because it sends the incoming messages to the Postmark Spam Check API.

Acknowledgements

Authors

Mathias Beke Timo Truyts

mistralmail's People

Contributors

denbeke avatar dependabot[bot] avatar michael-k avatar trtstm avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

mistralmail's Issues

Refactor handlers

We should make handlers cleaner and make objects of them, so they can call different functions on the same members and can store config data and a state for easier implementation.

Send \r\n instead of \n

Send \r\n after commands. We currently send \n but the spec says \r\n. And microsoft doesn't like \n...
We should also strip the \r\n from the commands we receive instead of only a \n.

Parser RCPT TO

als er geen email achter die TO komt returned de parser een RcptCmd. Zou eigenlijk een InvalidCmd moete returnen. InvalidCmd stuurt dan een SyntaxError naar de client. Denk eigenlijk dat het een SyntaxErrorParam moet zijn, maar daar hebbe wij geen cmd voor.

Edit: Er zijn wss nog zo'n gevallen

Blacklist could panic

Blacklist could panic on this:

func (ns *Nixspam) CheckIp(ip string) bool {
    index := sort.SearchStrings(ns.IpList, ip)
    return ns.IpList[index] == ip
}

index could equal len(ns.IpList).

SearchStrings searches for x in a sorted slice of strings and returns the index as specified by Search. The return value is the index to insert x if x is not present (it could be len(a)). The slice must be sorted in ascending order.

Config management

We need to work on how we store config in main.go to make it available for other component like e.g. the handlers.

What should UnknownCmd contain?

For example sending this command:

UNKN some unknown command

The parser creates this Command from it (note the \n): UnknownCmd{Cmd: "UNKN some unknown command\n"}.
I think it should not include the newline in the Cmd member. If this is changed we just have to change some expected values from the parser's tests.

Figure this out

This command keeps the server waiting for a command. Even if I type a newline manually.

echo -ne 'helo' | nc localhost 2525

Don't know if it's netcat or our server's problem.

Received-SPF

Next to Authentication-Results there is also the Received-SPF header to implement:

Received-SPF: fail (google.com: domain of [email protected] does not designate 137.191.225.35 as permitted sender) client-ip=137.191.225.35

RFC 4408 7. The Received-SPF Header Field

MTA/MSA/MUA

RFC 6409

About ports (abstract)

Abstract

This memo splits message submission from message relay, allowing each
service to operate according to its own rules (for security, policy,
etc.), and specifies what actions are to be taken by a submission
server.

Message relay is unaffected, and continues to use SMTP over port 25.

When conforming to this document, message submission uses the
protocol specified here, normally over port 587.

This separation of function offers a number of benefits, including
the ability to apply specific security or policy requirements.

About MTA and MSA (section 2.1)

Message Submission Agent (MSA)

A process that conforms to this specification. An MSA acts as a
submission server to accept messages from MUAs, and it either
delivers them or acts as an SMTP client to relay them to an MTA.

Message Transfer Agent (MTA)

A process that conforms to [SMTP-MTA]. An MTA acts as an SMTP server
to accept messages from an MSA or another MTA, and it either delivers
them or acts as an SMTP client to relay them to another MTA.

Message User Agent (MUA)

A process that acts (often on behalf of a user and with a user
interface) to compose and submit new messages, and to process
delivered messages.

For delivered messages, the receiving MUA may obtain and process the
message according to local conventions or, in what is commonly
referred to as a split-MUA model, Post Office Protocol [POP3] or IMAP
[IMAP4] is used to access delivered messages, whereas the protocol
defined here (or SMTP) is used to submit messages.

@trtstm

Check for args in helo and ehlo before using it.

Sending helo or ehlo without an address crashes the server.

// Uses args[0] without checking if there is an arg.
command = HeloCmd{Domain: args[0]}

Telenet gives this error when sending helo without an address:
501 HELO requires valid address

Avoid deadlock in parser.go

Check for error, otherwise we can get a deadlock. (Already fixed but not yet committed)

for line == "" {
    line, __ = br.ReadString('\n')
}

State reset before sending smtp.StartData

        state.reset()

        proto.Send(smtp.Answer{
            Status:  smtp.StartData,
            Message: "Start mail input; end with <CRLF>.<CRLF>",
        })

        data := []byte{}

Shouldn't be there since reset clears the from and rcpt fields.
Also use state.data instead of creating a new variable.

Parser probleem met data

Stel da ge deze cmds binnen krijgt:

helo
data
test mail
.

dan moet onze server na die data cmd al nen error geven dat er eerst een mail cmd moet zijn geweest.
Probleem is da de parser het geen dat erachter komt eerst parsed en dan pas een DataCmd returned.

We hebben zoiets nodig:

type DataCmd struct {
    data DeferedContent
}

type DeferedContent struct {
    br *bufio.Reader
}

func (c *DeferedContent) GetContents() string {
    // lees en return hetgeen dat achter het data cmd komt.
}

cmd.data.getContents()

Case sensitivity

We need tests with commands in upper/lower/mixed case.

RFC 5321 section 2.4.

Verbs and argument values (e.g., "TO:" or "to:" in the RCPT command
and extension name keywords) are not case sensitive, with the sole
exception in this specification of a mailbox local-part (SMTP
Extensions may explicitly specify case-sensitive elements). That is,
a command verb, an argument value other than a mailbox local-part,
and free form text MAY be encoded in upper case, lower case, or any
mixture of upper and lower case with no impact on its meaning. The
local-part of a mailbox MUST BE treated as case sensitive.
Therefore, SMTP implementations MUST take care to preserve the case
of mailbox local-parts. In particular, for some hosts, the user
"smith" is different from the user "Smith". However, exploiting the
case sensitivity of mailbox local-parts impedes interoperability and
is discouraged. Mailbox domains follow normal DNS rules and are
hence not case sensitive.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.