Check / verify / implement Postfix's and Dovecot's configuration for enabling (Perfect) Forward Secrecy.

References

• https://en.m.wikipedia.org/wiki/Forward_secrecy
• https://www.postfix.org/FORWARD_SECRECY_README.html
• https://www.heise.de/hintergrund/Forward-Secrecy-testen-und-einrichten-1932806.html
• https://www.heinlein-support.de/blog/security/perfect-forward-secrecy-pfs-fur-postfix-und-dovecot

Should not be needed, get rid of it...

Introduce rspamd into the setup.

For this step, simply make rspamd work.

• include rspamd from its own repositories
• adjust the default configuration slightly
  • make as much of its modules use redis as possible
  • deactivate real-time block lists (RBL)

Reference

• https://workaround.org/bullseye/filtering-out-spam-with-rspamd/
• https://www.rspamd.com/downloads.html#system2
• https://www.rspamd.com/doc/quickstart.html
• https://github.com/rspamd/rspamd

to be done

Verify and ensure, that Postfix and Dovecot are using journald for their logging.

The default workflow will perform various things, but at this point just perform a linting job.

• run black
• run isort
• run mypy
• run flake8
• OPTIONAL streamline this as make ci/linting ?
• include workflow badge in README

Relevant Dovecot settings

• 10-auth.conf
  • auth_mechanism: Outlook requires login, plain is the default
  • review the different authentication backends with Dovecot's documentation
• 10-mail.conf
  • mail_location
    • guide suggests maildir:~/Maildir, but as we are (mainly) concerned about virtual users, how does this work?
  • mail_plugins: add quota to manage the size of users' mailboxes
• 10-master.conf
  • provide the authentication socket required by Postfix:

    # Postfix smtp-auth
    unix_listener /var/spool/postfix/private/auth {
      mode = 0660
      user = postfix
      group = postfix
    }
    

  • create the lmtp socket for Postfix (might already be there, configure accordingly)

    service lmtp {
      unix_listener /var/spool/postfix/private/dovecot-lmtp {
        group = postfix
        mode = 0600
        user = postfix
      }
    }
    

  • review other settings here!
• 10-ssl.conf
  • ssl_cert and ssl_key: Point to the respective files
    • IMPORTANT syntax is like ssl_cert = </etc/letsencrypt/live/webmail.example.org/fullchain.pem, making the content of the certificate available!
• 20-lmtp.conf
  • if server side sieve should be applied, activate the plugin here (needs packet installation)
  • tracked in #21
• 90-quota.conf
  • Provide the actual mailbox quota

    plugin {
      quota = maildir:User quota
    
      quota_status_success = DUNNO
      quota_status_nouser = DUNNO
      quota_status_overquota = "452 4.2.2 Mailbox is full and cannot receive any more emails"
    }
    

  • provide the endpoint (socket) for Postfix

    service quota-status {
      executable = /usr/lib/dovecot/quota-status -p postfix
      unix_listener /var/spool/postfix/private/quota-status {
        user = postfix
      }
    }
    

    • Why is the definition of unix_listener lacking mode and group settings, as they are present in the other socket definitions?
  • the actual quota must be provided by the user_db; needs investigation!
  • Enable warnings about quotas:

    plugin {
       quota_warning = storage=95%% quota-warning 95 %u
       quota_warning2 = storage=80%% quota-warning 80 %u
    }
    service quota-warning {
       executable = script /usr/local/bin/quota-warning.sh  # Adjust script path!
       unix_listener quota-warning {
         user = vmail
         group = vmail
         mode = 0660
       }
    }
    

  • The referenced script to send the users emails about their quotas:

    #!/bin/sh
    PERCENT=$1
    USER=$2
    cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing"
    From: [email protected]
    Subject: Quota warning - $PERCENT% reached
    
    Your mailbox can only store a limited amount of emails.
    Currently it is $PERCENT% full. If you reach 100% then
    new emails cannot be stored. Thanks for your understanding.
    EOF
    

• Important somewhere is the user_db setting. This needs careful attention, because my setup will not rely on SQL as backend (deviating from the tutorial!); see https://doc.dovecot.org/configuration_manual/authentication/user_databases_userdb/ and https://doc.dovecot.org/configuration_manual/authentication/passwd_file/#authentication-passwd-file

Prerequisites

• Is the test suite working with the current, manual setup?
  • basically the test suite is working!
  • there are still some hard-coded assumptions in the test suite (mailsrv_aux.test_suite.smtp) and some configuration variables that are currently not exposed in test_runner.py; this have to be fixed!
• can a working Docker Container be created from our installation process?

Makefile includes code for a dedicated virtual environment for the test suite, but as of now, the test suite only relies on Python's standard library.

• remove code from Makefile
• remove requirements/test_suite.txt (see #4)
• remove requirements/test_suite.txt from tox configuration in pyproject.toml (this leaves a blank mypy for typechecking. However, the infrastructure to perform typechecking with the actual dependencies should be left in place. Document this somewhere!)

Provide a setup for sphinx, including a general structure for the documentation.

Pages

• Getting started
• Mail in a nutshell
• Configuration References
• Cookbook
  • Keep configuration in a dedicated directory or repository
    • Should be highly recommended!
  • In-depth setup of IP and DNS and how these things are working together
    • Including the guide on How to ensure Postfix uses the right IP
    • See #19
• Developer Documentation
  • Development Setup
  • Coding Style
  • Git Branching Model
• References
  • Official Documentation
  • Guides
  • Other Projects

As of now, the configuration templates have a .sample suffix. .template would be more fitting.

Requires adjustment of:

• Makefile
• .gitignore

Dovecot is used to authenticate (mail) users for itsself (IMAP/POP3) and for Postfix (SMTP submission).

In 10-auth.conf the setting auth_worker_max_count is heavily modified compared to Dovecot's default value of 30 (currently set to 5).

Assuming the number of possible virtual users / mailboxes, this might be sufficient for my personal setup, but probably this should be made configurable in settings.env or at least clearly documented.

While working on the configuration reference (as tracked in #23) it feels like duplicating too much.

configs/postfix/main.cf does already include lots of documentation, it feels unnecessary to duplicate this again for the Sphinx-based documentation.

Instead, let's parse the actual configuration files, extract the comments and generate the Sphinx documentation automatically.

For reference: 9f08744 has rST compatible documentation of some Postfix settings.

Include comments in the requirements files (see https://stackoverflow.com/questions/9159757/can-i-add-comments-to-a-pip-requirements-file):

• What is the purpose of the (specific) requirements file?
• Why are the versions not pinned? -> The requirements files are not actively used in the actual images, but provide repository-specific infrastructure around the images. Thus, pinning requirements to specific versions is not required.

mypy fails in GitHub Actions workflows sporadically. It can not be reproduced deterministically, so for now this is fixed by disabling the mypy pre-commit hook during CI.

This does introduce additional complexity to the Makefile.

There are quite a lot of (sub) services in Postfix's default master.cf. Probably some of them are not actively used in this setup.

Deactivating them seems desirable (resource usage, minimizing attack surface, ...)

A web-based, Django-based backend to manage virtual domains, mailboxes and aliases.

Just like my own, very old and outdated PHP-based project.

Once this issue will be worked on, it might be placed in a dedicated repository.

The steps to complete the CRAWL milestone (setting up Postfix and Dovecot) should be wrapped in a script for convenience.

• Use a combination of (small) bash scripts with Makefile as the controlling instance

• As of now, only the test suite and the related validator is implemented
  • Add the following block to all (Python) source files

    # SPDX-FileCopyrightText: 2022 Mischback
    # SPDX-License-Identifier: MIT
    # SPDX-FileType: SOURCE

• SPDX-FileType may be one of
  • https://spdx.github.io/spdx-spec/file-information/#8.3
  • SOURCE
  • DOCUMENTATION
  • OTHER

As of now, util/scripts/install-packages.sh provides the command to install the packages, including the actual list of packages.

This should be split into

• a script that includes the installation command (apt-get install)
• a list of required packages (might be placed in configs/ or requirements/

The setup should provide automatic filtering of mails, depending on user-defined rules.

The Python-based test suite should verify, that the general setup of the different components is working.

Prerequisites

• reference the actual mailserver (bare-metal or container setup) as mail.sut.test providing (virtual) mail addresses @sut.test

Test Cases

• SMTP (:25)
  • This is pretty straight forward: every mail is considered one test case, with an expected result.
  • TODO: Now we have parsing of the virtual_alias config, can we use this as an input to the SMTP test suite, to send a mail to every valid recipient?
  • The expected result is provided in a dictionary.

    • TODO: As of now, the implementation of SmtpTestSuite makes assumptions about the setup of the virtual alias database; this makes the whole test suite highly dependent on the actual test setup; this might be fixed by just returning a dict of the form

      result = {
          "mails_sent": [a, b, c, d, e, f, g],
          "mail_rejected": [e, f],
          "[email protected]": [a, c],
          "[email protected]": [c],
          "[email protected]": [b],
          "[email protected]": [d],
          "[email protected]":[g],
      }

      Then, the test runner tests/mailsrv_test_suite.py can merge the results of SmtpTestSuite and SmtpStarttlsTestSuite and process them by resolving the aliases.

    • TODO: The internal knowledge of the test setup is also reflected in the suite's logging messages. This should be fixed.

    • TODO: When the above two things are fixed, SmtpTestSuite may be simplified significantly!

  • send mail to [email protected]:
    • MAIL FROM: [email protected]
    • RCPT TO: [email protected]
    • message is accepted and placed in the (virtual) Maildir of [email protected]
  • send mail to [email protected]:
    • MAIL FROM: [email protected]
    • RCPT TO: [email protected]
    • message is accepted and placed in the (virtual) Maildir of [email protected]
  • send mail to [email protected] and [email protected]:
    • MAIL FROM: [email protected]
    • RCPT TO: [[email protected], [email protected]]
    • message is accepted and placed in the (virtual) Maildir of [email protected] and [email protected]
  • send mail to [email protected]:
    • MAIL FROM: [email protected]
    • RCPT TO: [email protected]
    • message is accepted and placed in the (virtual) Maildir of [email protected] and [email protected]
  • send mail to [email protected]:
    • MAIL FROM: [email protected]
    • RCPT TO: [email protected]
    • message is rejected because the user does not exist!
  • send mail to [email protected]
    • MAIL FROM: [email protected]
    • RCPT TO: [email protected]
    • message is rejected, because the server (SUT) does not relay mails without authentication
  • send mail to [email protected]
    • MAIL FROM: [email protected]
    • RCPT TO: [email protected]
    • message is accepted, but a bounce message (non delivery notification?) is emitted
  • use STARTTLS command
• POP3 (:110) / POP3s (:995)
  • Verify, that logins can only be performed over a secure connection (STARTTLS)
  • check the mailbox for expected messages
    • pick up and parse the protocol of the SMTP test suites, check all local mailboxes
• SMTP (submission :587)
  • Any "sending address" (might be determined from the LHS in util/test_configs/postfix_sender-login-map) should be able to send mail to local / internal accounts and external accounts
  • Sending of mails does require a valid login (RHS in util/test_configs/postfix_sender-login-map)
    • sending addresses may be usable by different login accounts (RHS of util/test_configs/postfix_sender-login-map might be a list); this has to be cross-checked!
    • actual logins have to be performed via a secure connection (TLS/STARTTLS)

Required Infrastructure

• [email protected] is an actual vmail user with a password in Dovecot's userdb
• [email protected] is an actual vmail user with a password in Dovecot's userdb
• [email protected] is a virtual alias to [email protected]
• [email protected] is a virtual alias to [email protected] and [email protected]
• [email protected] is a virtual alias to a non existent mailbox [email protected]
  • make sure [email protected] is not available in virtual_mailboxes and vmail_users
• [email protected] Make sure this account does not exist in Dovecot's userdb, Postfix's virtual_mailboxes database and Postfix's virtual_aliases database

The purpose of this repository changed!

The original idea was the development of a containerized mail setup, that was easy to manage and easy to deploy.

However, while testing and researching various stuff, I could not find easy solutions to all kind of problems. Furthermore, there are already several projects that do provide similar functionalities.

New Focus of the Repository

The ISPmail guide was the original blueprint for the desired mail setup. However, quite early during development, some things were changed, e.g. ditching the use of a SQL database for alias, mailbox and domain management.

However: As the need for containerization ceased (mostly inside of myself), going bare metal (or whatever you will call it on a VPS) brings database topic back into question, as the overall server will require a database server in any case.

Having said this: This repository is now intended to provide a ready to go configuration template, so that the mail service (inlcuding Postfix, Dovecot, ...) can be implemented with very little effort.

Everything HTTP-related, including the webmail interface and the generation/renewal of SSL certificates will be skipped aswell.

This issue just documents the configuration of Postfix with some additional notes.

Please note that Postfix and Dovecot work closely together and can not be handled independently!

Postfix Configuration

• mydestination for Postfix's canonical domains, that is domains that are directly related to the server. Might just be localhost and the actual mail setup relies on virtual domains
• mynetworks: Should most likely be empty or just containing my very own subnet
• Use Dovecot for SASL authentication: smtpd_sasl_type=dovecot
• Use the existing authentication socket: smtpd_sasl_path=private/auth
• smtpd_sasl_auth_enable=yes Verify this setting
• smtpd_tls_security_level=may Verfiy this setting (document what it is doing!)
• smtpd_tls_auth_only=yes Verfiy this setting (document what it is doing!)
• smtpd_tls_cert_file=/etc/letsencrypt/live/webmail.example.org/fullchain.pem
  • provide the actual path to the certificate file
• smtpd_tls_key_file=/etc/letsencrypt/live/webmail.example.org/privkey.pem (see above)
• smtp_tls_security_level=may Verify this setting (document what it is doing!)
• virtual mailboxes: virtual_mailbox_maps
• virtual domains: virtual_mailbox_domains
• (virtual) aliases: virtual_alias_maps
• Verify: smtpd_recipient_restrictions=reject_sender_login_mismatch
• use Dovecot's lmtp service: virtual_transport=lmtp:unix:private/dovecot-lmtp
• use Dovecot's quota service: smtpd_recipient_restrictions=reject_unauth_destination,"check_policy_service unix:private/quota-status"
• smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination Verify this setting (document what it is doing!)
• enable submission service in /etc/postfix/master.cf

  submission inet n       -       y       -       -       smtpd
   -o syslog_name=postfix/submission
   -o smtpd_tls_security_level=encrypt
   -o smtpd_sasl_auth_enable=yes
   -o smtpd_tls_auth_only=yes
   -o smtpd_reject_unlisted_recipient=no
   -o smtpd_recipient_restrictions=
   -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
   -o milter_macro_daemon_name=ORIGINATING
  

• review smtpd_sender_login_maps and make that work
  • The tutorial allow (by default) only sending from your own account. The setup should support sending from alias addresses aswell
  • Has to be included in the submission service like -o smtpd_sender_restrictions=reject_sender_login_mismatch,permit_sasl_authenticated,reject
  • needs mirror rule: [email protected] [email protected]
• inet_interfaces=all Verify this setting! Details in #19

rspamd relies on redis as a fast caching backend.

This issue is directly related to #15.

Requirements

• install redis from Debian/stable
• provide a basic configuration
  • only available locally (bind to localhost)
  • limit RAM usage (needs investigation)
  • write to disk frequently (needs investigation)
• Can redis be used for other required caching?
  • Postfix?
  • nginx
  • Django

Research, evaluate and then document IP address-related settings of Postfix.

Relevant Settings

• inet_interfaces
• smtp_bind_address
  • not tested
  • assumed to be working like smtp_bind_address6
• smtp_bind_address6
• smtp_bind_address_enforce
  • available from Postfix > 3.7
  • not tested, as Debian/stable (bullseye) only includes 3.5
  • left in main.cf.sample with a clarifying comment
• inet_protocols
  • not tested
• smtp_address_preference
  • not tested

Required Test Cases

• inet_interfaces set to a specific IP
  • Is that IP used for all inbound and outbound connections?
  • This would be the most obvious and easy configuration
• inet_interfaces set to a specific IP + localhost
  • Is the IP used for all inbound and outbound connections? At least the external ones?
  • This would be the most desirable configuration (easy, obvious, functional)
• inet_interfaces set to all in combination with smtp_bind_address6
  • Is smtp_bind_address6 used for all outbound connections?!
• inet_interfaces set to all in combination with smtp_bind_address6 and multiple available IPv6 interfaces.
  • Verify smtp_bind_address_enforce. I feel like this should be set to yes.

Notes

• if inet_interfaces is set to an actual IP address (v4 or v6), Postfix binds itsself only to that interface (see ss -lntp output)
• inet_intonly to the specified interfaceserfaces might be a list, e.g. 192.168.100.5, 127.0.0.1, [::1]
• if inet_interfaces is set to all and inet_protocols is set to ipv6, Postfix binds itsself to IPv6 interfaces only (see ss -lntp output)

Original Issue Statement

Postfix might be tied to a dedicated IPv4/IPv6 address.

This will make SPF/DMARC/... possible without too much interdependent settings with other services.

Highly relevant for an IPv6 setup, that maintains scalability.

See https://serverfault.com/questions/92181/how-to-make-postfix-use-another-ip-address#92207

Originally posted by @Mischback in #13 (comment)

Follow up to #1

Create Test Suite to verify the quota setup

The configuration applies a general quota to all mailboxes. User-specific quotas may be applied using dovecot's userdb (passwd-like file).

The server automatically notifies the user about quota limits (as of now at 80% and 95% of the quota).

The test suite should check for these notifications.

Implementation Idea

• use a dedicated user / mailbox for the test
• limit capacity to a known (and easily reachable) value (e.g. 1kB)
• manually reach a limit of 80% of the capacity
• manually reach 95% of the capacity
• add notification mails to expected_messages that will be checked during the (POP3-based) testing stage

Provide a minimal dependabot setup

• Dependency Management of Python dependencies
• protect the default branch (see one of my NodeJS repositories)
• automatic approval of PR for MINOR/PATCH updates (see one of my NodeJS repositories)

SPF is not really part of the (technical) server setup, as it is defined in the host's/domain's DNS setup.

However, it should be considered while setting up the mail server.

Idea

• Include check for a valid SPF DNS record in the validator script

References

• https://www.spfwizard.net/
• https://workaround.org/bullseye/prevent-spoofing-using-dkim-2/
• https://thomas-leister.de/mailserver-debian-buster/#spf-records

• Configuration Verification
  • read relevant configuration files
    • Dovecot's userdb: vmail_users
    • Postfix's virtual_mailboxes
    • Postfix's virtual_aliases
    • ...
  • perform parsing and tests
    • all entries in Postfix's virtual_mailboxes must have a corresponding entry in Dovecot's userdb (vmail_users)
    • all of Postfix's addresses (virtual_mailboxesand the left-hand side of virtual_aliases) must have a domain part in virtual_domains
    • all entries in Postfix's virtual_aliases must resolve to an entry in virtual_mailboxes (or an external address, which raises another warning!)
    • all entries in Postfix's virtual_aliases should not resolve to an external address
    • all entries in Postfix's virtual_mailboxes combined with virtual_aliases should have a corresponding entry in sender_login_map
    • all entries in Postfix's sender_login_map must resolve to an entry in Dovecot's userdb
    • all entries in Dovecot's userdb should have one of the following:
      • a (left hand side) entry in virtual_mailboxes
      • a (right hand side) entry in sender_login_map
    • all domains in virtual_domains should have postmaster@domain and abuse@domain addresses (mailbox or alias)
    • Dovecot's userdb should not contain plain text passwords ({plain}foobar)