Whitelist ignored about mailcow-dockerized HOT 16 CLOSED

Thanks to @broedli we could fix it. :-)

Will be merged from master to dev today.

from mailcow-dockerized.

from mailcow-dockerized.

Thanks for the reply and sorry for my late answer.

It looks like there is the same problem with the blacklist. The cache probably doesn't cause the problem as I restarted/rebuilt several times and now - some days, not only minutes later ;-) - the problem still persists.

The full output of the settings file would be like really long plus contain a lot of addresses of our users but here the rule of the user in question in the full hierarchical structure of the file. So I removed other score_, whitelist_ and blacklist_ rules and replaced the users alias addresses (originally 61 in this case) with [email protected]. The original output had no missing brackets or other noticeable syntax issues:

settings {
        whitelist_userdomainde {
                priority = high;
                from = "/(.*@gmail.com|.*@gmail.com.*|.*@googlemail.com|.*@googlemail.com.*)/";
                rcpt = "[email protected]";
                apply "default" {
                        MAILCOW_MOO = -999.0;
                }
        }
}

We do have a volume in rspamd which I think you don't have currently for /var/lib/rspamd/ to persist dynamically added rules / configuration changes via web GUI. Could this cause this problem somehow? I'm not sure where/how the output of the settings file should be applied in rspamd or rather would it be written to a file or is it just loaded directly into it's memory?

from mailcow-dockerized.

Can you grab the latest settings.php? Do the Rspamd logs contain something along "61 settings loaded from xy"? What does the score setting look like for this user?

from mailcow-dockerized.

I just pulled the latest images and merged the latest updates including settings.php (I thought I was quite up to date but it looks like I wasn't or there were a lot of updates the last 2 weeks or so ;-)).
This seems to have worked well for everything but rspamd which now seems to completely not work and I cannot see why atm. Its webinterface is offline and the log looks like this:

rspamd-mailcow_1     | 2017-01-30 16:33:17 #1(main) <a32e06>; main; load_rspamd_config: tempdir is not set, trying to use $TMPDIR
rspamd-mailcow_1     | 2017-01-30 16:33:17 #1(main) <a32e06>; main; load_rspamd_config: $TMPDIR is empty too, using /tmp as default
rspamd-mailcow_1     | 2017-01-30 16:33:21 #1(main) <srkwm6>; symcache; rspamd_symbols_cache_post_init: cannot register delayed condition for DMARC_POLICY_ALLOW
rspamd-mailcow_1     | 2017-01-30 16:33:21 #1(main) <srkwm6>; symcache; rspamd_symbols_cache_post_init: cannot register delayed condition for R_SPF_ALLOW
rspamd-mailcow_1     | 2017-01-30 16:33:21 #1(main) <srkwm6>; symcache; rspamd_symbols_cache_post_init: cannot register delayed condition for R_DKIM_ALLOW
rspamd-mailcow_1     | 2017-01-30 16:33:21 #1(main) <srkwm6>; symcache; rspamd_symbols_cache_post_init: cannot find dependency on symbol FREEMAIL_REPLYTO
rspamd-mailcow_1     | 2017-01-30 16:33:21 #1(main) <srkwm6>; symcache; rspamd_symbols_cache_post_init: cannot find dependency on symbol FREEMAIL_FROM
rspamd-mailcow_1     | 2017-01-30 16:33:21 #1(main) <srkwm6>; symcache; rspamd_symbols_cache_validate: symbol 'HFILTER_HELO_2' has its score defined but there is no corresponding rule registered

And the last line repeats for I guess every symbol which is configured and then the log stops. Mails go through however without rspamd classification / headers and without DKIM signing. So I also cannot troubleshoot the original problem right now and am not sure if it still would be there...
Any quick thoughts on this? Thx in advance ;-)

from mailcow-dockerized.

from mailcow-dockerized.

The volumes should be like yours right now and when I docker exec into the container as _rspamd I see everything under /etc/rspamd/ and also have read access to it...

from mailcow-dockerized.

from mailcow-dockerized.

/etc/rspamd/modules.d/ is and everything inside it also readable by _rspamd

from mailcow-dockerized.

My interim problem with rspamd was caused by some config files with Windows line endings which got mixed in during the update somehow. Looks like rspamd really does not like Windows :p

Anyways, back to the original problem

Can you grab the latest settings.php? Do the Rspamd logs contain something along "61 settings loaded from xy"? What does the score setting look like for this user?

I now have the latest settings.php and the same problem. When I do a docker-compose logs rspamd-mailcow |grep settings there are only two lines (see below) and nothing user specific like you were looking for. And this user has no other settings, only the whitelist_.

rspamd-mailcow_1     | 2017-01-30 22:32:34 #1(main) <oxhd99>; map; rspamd_map_add: added map http://nginx:8081/settings.php
rspamd-mailcow_1     | 2017-01-30 22:32:34 #1(main) <srkwm6>; cfg; rspamd_init_lua_filters: init lua module settings

from mailcow-dockerized.

There is one thing in the log that stands out a little bit though:

ERROR:/rspamd-1.4.3/src/libutil/map.c:473:http_map_finish: assertion failed: (cbd->shmem_data != NULL)

Just from the words map and http I could imagine this has something to with the problem :d

from mailcow-dockerized.

from mailcow-dockerized.

At this point I'm not sure anymore if the error about the assertion failure is the problem. And I'm quite confused as this problem isn't behaving consistent at all. I could reproduce it partly on an entire other server and fresh non-modified docker-mailcow setup with one domain.
The full and unmodified settings output there is:

settings {
	score_testtestde {
		priority = low;
		from = "/^((?!.*@gmail.com).)*$/";
		rcpt = "[email protected]";
		apply "default" {
			actions {
				reject = 13;
				greylist = 6.5;
				"add header" = 7.5;
			}
		}
	}
	whitelist_testtestde {
		from = "/(.*@gmail.com)/";
		priority = high;
		rcpt = "[email protected]";
		apply "default" {
			MAILCOW_MOO = -999.0;
		}
	}
}

To ensure this is not the problem I changed the the settings.php on the live system to output the exact same. Then I took the valid source of a gmail.com e-mail and changed its recipient to [email protected]. In the rspamd Web GUI I scanned this source on both systems with results that kinda looked like this:

On the test system with the current unmodified mailcow version:

• There also sometimes is the assertion failed error
• It lookes like most of the times assertion failed comes up when docker-compose restart rspamd-mailcow is performed individually and not docker-compose restart
• In ca. 1 out of 10 times after a restart the settings (and thus the whitelist) are ignored and this is permanent till the next restart (sometimes immediately after restart the whitelist seems not to be loaded and is ignored but it will be used after 30 to 60 seconds or so)
• It seems that often when the settings are ignored the assertion failed error is there, however there also are times were assertion failed is present and the settings do work

On the live system:

• Sometimes (not often) there is no assertion failed error
• In ca. 1 out of 10 times after a restart the settings are not ignored and actually work
• I had one case where the settings were working and weren't hard coded to the above output yet. So this were the actual settings with the full long list...
• I can definitely also happen that assertion failed is present and the rules are still working (however in that case I think the assertion failed refers to some rspamd.com map)

During these restart tests I didn't modify any files or rules...

I'm really a little bit clueless now, maybe it has something to do with the container start order? As workaround I now at least can restart until the settings work :D

from mailcow-dockerized.

You use a custom docker-compose file, right? Do you have dependencies set?

from mailcow-dockerized.

On the live system yes but it only differs in volumes and drivers not in dependencies.
And on the test system I followed the exact install procedure from https://mailcow.email/dockerized/?c=I.%20Installation&p=Install_mailcow_dockerized.md, logged in as Admin and created the domain test.de (litterally test.de so this is no example) and the Mailbox [email protected]. Logged in as [email protected] and created the score and whitelist rules to get the output you see above. After that I just set a rspamd Web GUI password and tested with the "Scan" feature from the Web GUI. And as I could reproduce all problems and errors I see in the live system here but with zero modifications (also no docker-compose.yml modifications) I don't think that is the problem.

Right now my probability to get a working state on the test system also was closer to the live system. So it took a lot more restarts/recreates until the rules worked. (And I again changed servers for the test system, now I was on desktop Mint installation while the others were always Ubuntu server installations)

When the rules work there are always at least the following two log entries present:

rspamd-mailcow_1     | 2017-02-01 15:52:51 #13(normal) <5tzsu9>; map; http_map_finish: read map data from nginx (16 bytes)
rspamd-mailcow_1     | 2017-02-01 15:52:54 #8(normal) <srkwm6>; lua; settings.lua:503: loaded 2 elements of settings

Those never exist when the rules don't work.

If you find the time I think you would also be able to reproduce by creating such a setup and restarting a couple of times. And if you want a test source (which probably wouldn't even be necessary as you can see from the log entries if the settings get loaded or not) you can use:

// Update: If you try to reproduce the problem I think it is a good idea to mix up individual rspamd-mailcow restarts and restarts of all containers - also I couldn't see any consistency regarding the restart type as well...

Return-Path: <[email protected]>
Received: from mail.somedomain.de ([172.22.1.100])
	by mail.somedomain.de (Dovecot) with LMTP id eGseDdxekFhgAAAAw+ZHhw
	for <[email protected]>; Tue, 31 Jan 2017 15:21:28 +0000
Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54])
	by mail.somedomain.de (Postfix) with ESMTPS id 6A14E2271D5
	for <[email protected]>; Tue, 31 Jan 2017 15:21:27 +0000 (UTC)
Received: by mail-wm0-f54.google.com with SMTP id b65so84554916wmf.0
        for <[email protected]>; Tue, 31 Jan 2017 07:21:27 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20161025;
        h=from:subject:to:message-id:date:user-agent:mime-version;
        bh=YxN1f9UjTnkc9T+3IN7JKQoLXUUmr4aYG58i/f3KlJc=;
        b=kPamtiInhuxP2U7q6/P79wxOf2lYDBpVMuTpcBXfaRN88r1JxdYuvt9lxviIE2BsVp
         LPu9AAzaHDu8YHyIEv58/n33vVfkX2gfMLEU8jER9b6W6lMAFjn7kBln+4d7UTHLK0Sg
         4BpDJ1McXFhY776rYzhU8p1kqbLe0ubED2JFd5/t2FoIK35hZPzHt1jSKOzm2y8thD5W
         ZHEUcRimLTz7fGPJY2AEZEBCys1rax84jvGy5oN3tQBQ8fI5Mn74jbTofj4cwmG7z3M1
         gEk/1G5F2a/P03sMyeS3Tl67l2lXxR7neqL7DDq32BySAtxnbFn/gndgnBBk0ZoSA1F+
         vPRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20161025;
        h=x-gm-message-state:from:subject:to:message-id:date:user-agent
         :mime-version;
        bh=YxN1f9UjTnkc9T+3IN7JKQoLXUUmr4aYG58i/f3KlJc=;
        b=nBaD0XyWbIcgn2ZXr80bGQ9RAasJiAuOFN3WZlnWuWdIIX6CakMe4PeaX9ZozS6TZO
         ph5IdqSqolzK6SafAj8EweAHbbg14i9+3El1FXrGRaPDR6EkpaIB3V4xRmm5JejUmaRY
         5dNeSiusFAhvKu3+HofRPpoxJAG2kKRgGTsX/2OX0MFxbhHPa5qbt+RYSTRLqGz1wrae
         2tUGrJIAuT3Q9HSRZnCnKNKo2gN1zupiBn1bOkjmDZkEbquyeEwCiDo8tkStvxnxLwgS
         UfpWAxXd429YK2FxF6l3awf4FN5pfNpCcYuiuweaKoNIbnsxbj7ebDR8kWQC6lIL+n23
         RTJQ==
X-Gm-Message-State: AIkVDXJx6gjWd7AyRvyYpQ3jRKu0Uweskg9FnKE0RGrhjuSFVFtUTE4ZHidO1877Klv8sQ==
X-Received: by 10.28.183.5 with SMTP id d7jr53562693wmf.39.2184876763272;
        Tue, 31 Jan 2017 07:21:26 -0800 (PST)
Received: from [192.168.1.77] (fttx-55.55.55.55.someprovider.de. [55.55.55.55])
        by smtp.googlemail.com with ESMTPSA id z67sm29088743wrb.49.2017.01.55.55.55.55
        for <[email protected]>
        (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
        Tue, 31 Jan 2017 07:21:25 -0800 (PST)
From: Someone <[email protected]>
X-Google-Original-From: Someone <[email protected]>
Subject: Test
To: "Test" <[email protected]>
Message-ID: <[email protected]>
Date: Tue, 31 Jan 2017 16:21:24 +0100
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.16)
 Gecko/20080708 Lightning/0.8 Thunderbird/2.0.0.16 Mnenhy/0.7.5.666
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="------------6155775E7EB380E3FAB93786"

This is a multi-part message in MIME format.
--------------6155775E7EB380E3FAB93786
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

Test

--------------6155775E7EB380E3FAB93786
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <font face="Helvetica, Arial, sans-serif">Test</font>
  </body>
</html>

--------------6155775E7EB380E3FAB93786--

This should score with -997.60 or 1.40.

from mailcow-dockerized.

Awesome! I nearly got used to the restart routine but am really happy about the real fix :)

from mailcow-dockerized.