bug report: I copied volumes to another system to migrate DMS but it fails to start properly about docker-mailserver HOT 8 CLOSED

Please note that this response was created using generative AI, leveraging previously available information from the repository. This information may be helpful. However, it is our strong recommendation that you carefully consider the information before blindly applying it.

The issue you're encountering with permission denials, especially related to Amavis and Postfix, is likely due to incorrect ownership or permissions on the directories and files used by these services. This can happen when moving data between servers or reinstalling the OS, as the user and group IDs might not match.

Here are some steps to resolve the issue:

1. Ensure Correct Ownership and Permissions:

   • Check the ownership and permissions of the directories and files within /var/mail/, /var/mail-state/, and /tmp/docker-mailserver/. These should be owned by the user and group that the mail services run as, which is typically postfix or amavis.
   • You can adjust the permissions with commands like:

     chown -R 110:112 /var/mail/
     chown -R 110:112 /var/mail-state/
     chown -R 110:112 /tmp/docker-mailserver/

     Replace 110:112 with the correct user and group IDs for your setup.

2. Restart Services:

   • After adjusting the permissions, restart the Docker container to ensure all services can start with the correct permissions:

     docker-compose down
     docker-compose up -d

3. Check Container Logs:

   • After restarting, check the logs again to ensure that the permission issues are resolved:

     docker logs mailserver

4. Use Docker Volumes:

   • To avoid such issues in the future, consider using Docker volumes for persistent data storage. Docker manages the permissions and ownership of volumes automatically, which can help prevent such issues when moving containers between hosts.

5. Documentation and Resources:

   • Review the Docker Mailserver documentation related to debugging and permissions.

If these steps do not resolve the issue, you may need to provide more detailed logs or configuration details to further diagnose the problem.

This comment was generated by Glime.

from docker-mailserver.

services:
  mailserver:
    image: ghcr.io/docker-mailserver/docker-mailserver:latest

You're sure the version of DMS container was the same between both? :latest doesn't help ensure that if the prior system :latest was an older release.

move ./docker-data/, mailserver.env, docker-compose.yml to another server (or reinstall os) and start compose

You can't just copy/paste and expect that to work.

We have some guidance on our docs FAQ for backup/restore. Using tar you can preserve the metadata like ownership + permissions. rsync can do this too, but plain copy/paste won't.

[  DEBUG  ]  Enabling and configuring Rspamd
[ WARNING ]  (Rspamd setup) Running Amavis/SA & Rspamd at the same time is discouraged
[ WARNING ]  (Rspamd setup) Running OpenDKIM & Rspamd at the same time is discouraged - we recommend Rspamd for DKIM checks (enabled with Rspamd by default) & signing
[ WARNING ]  (Rspamd setup) Running OpenDMARC & Rspamd at the same time is discouraged - we recommend Rspamd for DMARC checks (enabled with Rspamd by default)
[ WARNING ]  (Rspamd setup) Running policyd-spf & Rspamd at the same time is discouraged - we recommend Rspamd for SPF checks (enabled with Rspamd by default)

You probably should consider these warnings?

[  DEBUG  ]  Consolidating all state onto /var/mail-state
[  DEBUG  ]  Checking /var/mail permissions
find: ‘standard output’: Broken pipe
find: write error
[  DEBUG  ]  Removing files and directories from older versions
[  DEBUG  ]  Ensuring /tmp/docker-mailserver/rspamd/dkim is owned by '_rspamd:_rspamd'
[  DEBUG  ]  Exporting environment variables now (creating '/etc/dms-settings')
chown: cannot access '/tmp/docker-mailserver/rspamd/dkim': No such file or directory

May  9 14:29:04 mail postfix/postfix-script[1001]: fatal: Postfix integrity check failed!
May  9 14:29:04 mail amavis[939]: starting. /usr/sbin/amavisd-new at mail.blackrandcrf.ru amavisd-new-2.11.1 (20181009), Unicode aware, LC_CTYPE="C.UTF-8"
May  9 14:29:04 mail amavis[939]: perl=5.032001, user=, EUID: 110 (110);  group=, EGID: 112 112 (112 112)
May  9 14:29:04 mail amavis[939]: (!)Net::Server: 2024/05/09-14:29:04 Can't connect to UNIX socket at file /var/lib/amavis/amavisd.sock [Permission denied]\n  at line 66 in file /usr/share/perl5/Net/Server/Proto/UNIX.pm
May  9 14:29:05 mail amavis[1002]: starting. /usr/sbin/amavisd-new at mail.blackrandcrf.ru amavisd-new-2.11.1 (20181009), Unicode aware, LC_CTYPE="C.UTF-8"

• Not sure about the find issue with /var/mail.
• chown for /tmp/docker-mailserver/rspamd/dkim should be fairly straight-forward, file/dir doesn't exist? (this may actually be a bug that was fixed in upcoming :14 release (:edge tag would confirm_).
• The remaining log errors are related to /var/mail-state, which you probably don't need to keep, you can comment out the mount to try and if those errors disappear (they should) then you can restore the mount but empty out the local directory content. If you are concerned, if possible you can take the original files from the prior system and do the backup/restore properly which should preserve the permissions.

That said the /var/mail-state should have it's permissions corrected at runtime to avoid that. So something else might be wrong with how you approached this that prevents DMS from doing things as root that it normally should be able to do. Potentially we may have missed something in our own scripts, but I'm not sure where.

from docker-mailserver.

You're sure the version of DMS container was the same between both? :latest doesn't help ensure that if the prior system :latest was an older release.

i have old logs, but i can't find dms version from files

You probably should consider these warnings?

i check later
upd: fixed

If you are concerned, if possible you can take the original files from the prior system and do the backup/restore properly which should preserve the permissions.

i can't take original files from prior server because i reinstalled os on server

from docker-mailserver.

i can't take original files from prior server because i reinstalled os on server

I understand. In future it's best to test that a backup restores properly to avoid this sort of situation 😓

Following the advice I gave above should fix it, it's unlikely you have anything important in /var/mail-state volume, it's for the services at runtime that's usually not important to persist, but sometimes helpful for stuff like Postfix queue not being lost, and I believe other data like Redis database for rspamd is also managed here if you're training spam detection.

Anything specific to DMS is via our config volume (/tmp/docker-mailserver) and for your mail data /var/mail. These two are more important to backup and restore correctly. For /var/mail try chown -R 5000:5000 on the local folder.

from docker-mailserver.

I'm getting these errors right now after

root@quixotic-toothpaste:~/mail# chown -R 5000:5000 docker-data/*
root@quixotic-toothpaste:~/mail# chown -R 5000:5000 /var/mail/*
chown: cannot access '/var/mail/*': No such file or directory
root@quixotic-toothpaste:~/mail# chown -R 5000:5000 /var/mail/

...
mailserver  | [   INF   ]  mail.blackrandcrf.ru is up and running
mailserver  | May 11 07:51:21 mail postfix/postfix-script[922]: fatal: Postfix integrity check failed!
mailserver  | May 11 07:51:24 mail postfix/postsuper[975]: fatal: scan_dir_push: open directory defer: Permission denied
mailserver  | May 11 07:51:25 mail postfix/postfix-script[976]: fatal: Postfix integrity check failed!
mailserver  | May 11 07:51:31 mail postfix/postsuper[1033]: fatal: scan_dir_push: open directory defer: Permission denied
mailserver  | May 11 07:51:32 mail postfix/postfix-script[1037]: fatal: Postfix integrity check failed!
mailserver  | May 11 07:51:38 mail postfix/postsuper[1093]: fatal: scan_dir_push: open directory defer: Permission denied
mailserver  | May 11 07:51:39 mail postfix/postfix-script[1094]: fatal: Postfix integrity check failed!

from docker-mailserver.

I'm getting these errors right now after

You need to run the command on the local location you have for the containers /var/mail mount. In your case the mail-data/ folder.

from docker-mailserver.

You need to run the command on the local location you have for the containers /var/mail mount. In your case the mail-data/ folder.

error still here :(
folders perms:

root@quixotic-toothpaste:~/mail/docker-data/dms# chown -R 5000:5000 ./*
root@quixotic-toothpaste:~/mail/docker-data/dms# ls -lh
total 16K
drwxrwx---  3 5000 5000 4.0K May 11 09:48 config
drwxrwx---  4 5000 5000 4.0K May  8 20:50 mail-data
drwxrwx---  2 5000 5000 4.0K May  8 20:50 mail-logs
drwxrwx--- 10 5000 5000 4.0K May  8 20:51 mail-state

after starting containers perms:

root@quixotic-toothpaste:~/mail# ls ./docker-data/dms/ -lh
total 16K
drwxrwx---  3 5000 5000 4.0K May 11 09:59 config
drwxrwx---  4 5000 5000 4.0K May  8 20:50 mail-data
drwxrwx---  2  115 root 4.0K May  8 20:50 mail-logs
drwxrwx--- 10 5000 5000 4.0K May  8 20:51 mail-state

from docker-mailserver.

error still here :(

You need to pay attention...

• ONLY mail-state should have been adjusted to 5000:5000 which is /var/mail within the container.
• mail-logs (maybe?), mail-state and config probably all need to be corrected. config should mostly be root (0), but I think some files may vary 🤷‍♂️
• Most of mail-state should have been corrected by DMS at runtime. Did you try running with this volume commented out in compose.yaml? It was suggested.

Make a copy of this docker-data/dms/ folder somewhere else.

• Try disabling all of the mounts, the container should run like it's new and say you have 120 seconds to create an account. Create one and confirm everything starts correctly.
• Now docker compose down, add the mail-data volume back in compose.yaml, try running DMS again, everything good?
• Now docker compose down again and bring back the config volume in compose.yaml. Correct the ownership first with chown -R 0:0 config and that should cover most of the expected ownership.
• Run DMS with this config volume and mail-data, everything still good? If so try to use it properly.
• If you're happy with what you've got at this point, keep a backup of mail-logs and mail-state if you like for a little bit, but otherwise you can ignore these and remove them once you're sure there's nothing wrong.

Try to follow those steps properly. I'm not able to continue providing personal support like this and will direct you to our Discussions where you can try engage with the community for further support if you need it 👍

from docker-mailserver.