Giter Club home page Giter Club logo

postfwd-anti-geoip-spam-plugin's Introduction

Table of Contents

Postfwd GeoIP Botnet Block Plugin

This is a plugin to postfix firewall postfwd (also located on github) intended to block international spam botnets. International spam botnets are logging into hacked mail addresses via sasl login from multiple IP addresses based in usually more than 30 unique countries. After successful login, the hackers send spam from huge amount of unique IP addresses which circumvents traditional rate limits per IP address.

If you are interested in theory about how botnet spam works and motivation for creating this plugin, look at the blog on Medium.

If you are interested in how your users got their mail accounts hacked, check out bsdly blog about slow distributed brute force attack on SSH passwords, which also applies to pop3/imap logins Hail Mary Cloud.

Plugin Compatibility Matrix

Plugin Version Postfwd Version GeoIP Version IP version
v2.0.0 postfwd3 v2.xx GeoIP 2 IPv4, IPv6
v1.50.0 postfwd3 v2.xx GeoIP 1, 2 IPv4
v1.40 postfwd3 v2.xx GeoIP 1 IPv4
v1.30 postfwd3 v2.xx GeoIP 1 IPv4
v1.21 postfwd1, postfwd2 v1.xx GeoIP 1 IPv4
  • Supported database backends are MySQL and PostgreSQL.

To list changed between versions check release notes or look into the Changelog.

Running with Docker

Pre-built ready-to-use Docker image is located on DockerHub and can be simply pulled by command:

# postfwd3 tags
docker pull lirt/postfwd-anti-geoip-spam-plugin:v2.0.0
# postfwd1, postfwd2 tags
docker pull lirt/postfwd-anti-geoip-spam-plugin:v1.21

To run postfwd with geoip-plugin, run docker with configuration files mounted as volumes:

docker run \
    -v </absolute/path/to/anti-spam.conf>:/etc/postfwd/anti-spam.conf \
    -v </absolute/path/to/postfwd.cf>:/etc/postfwd/postfwd.cf \
    lirt/postfwd-anti-geoip-spam-plugin:v2.0.0

This will run postfwd2 or postfwd3 (based on docker tag) with default arguments, reading postfwd rules file from your mounted volume file postfwd.cf and using anti-spam configuration from your file anti-spam.conf.

Installation

To install this plugin follow next steps:

  • Clone this repository.
  • Install dependencies according to chapter Dependencies.
  • Run script install.sh to install plugin into /etc/postfix/.
  • To load plugin you must add argument --plugins <PATH TO PLUGIN> to postfwd command (or update it in /etc/default/postfwd).
  • Configure postfwd rules according to chapter Postfwd configuration.
  • Create database table with indexes using following SQL statements (database is created on plugin startup but indexes cannot be).
CREATE TABLE IF NOT EXISTS postfwd_logins (
   sasl_username varchar(100),
   ip_address varchar(45),
   state_code varchar(4),
   login_count int,
   last_login timestamp
);
CREATE INDEX postfwd_sasl_client_state_index ON postfwd_logins (sasl_username, ip_address, state_code);
CREATE INDEX postfwd_sasl_username ON postfwd_logins (sasl_username);

Dependencies

  • Postfwd2 or Postfwd3.
  • Database (MySQL or PostgreSQL).
  • Perl modules - Geo::IP, DBI, Time::Piece, Config::Any, Net::Subnet, DBD::mysql or DBD::Pg.
  • GeoIP database (GeoIP version 1 in releases before 2.0.0 or GeoIP version 2 since release 2.0.0).

Cpanm

You can install all dependencies using cpanm with single command cpanm --installdeps .

Dependencies on RedHat based distributions

Install GeoIP, Time, Config, DBI and database modules with following command:

yum install -y 'perl(Geo::IP)' \
               'perl(Time::Piece)' \
               'perl(Config::Any)' \
               'perl(DBI)' \
               'perl(DBD::mysql)' \
               'perl(DBD::Pg)' \
               'perl(Net::Subnet)' \
               'perl(GeoIP2::Database::Reader)' \
               'perl(Net::SSLeay)' \
               'perl(IO::Socket::SSL)' \
               'perl(LWP::Protocol::https)' \
               'perl(Class::XSAccessor)' \
               'perl(MaxMind::DB::Reader::XS)' \
               'perl(Readonly)' \
               'perl(Data::Validate::IP)'

Dependencies on Debian based distributions

Install GeoIP, Time, Config, DBI and database modules with following command:

apt-get install -y libgeo-ip-perl \
                   libtime-piece-perl \
                   libconfig-any-perl \
                   libdbi-perl \
                   libdbd-mysql-perl \
                   libdbd-pg-perl \
                   libnet-subnet-perl \
                   geoip-database \
                   libnet-ssleay-perl \
                   libio-socket-ssl-perl \
                   liblwp-protocol-https-perl \
                   libclass-xsaccessor-perl \
                   libmaxmind-db-reader-xs-perl \
                   libgeoip2-perl \
                   libreadonly-perl \
                   libdata-validate-ip-perl

Configuration

Plugin configuration file anti-spam.conf is INI style configuration file, in which values must NOT be quoted!

Postfwd configuration

Add following rules to postfwd configuration file postfwd.cf. You can use your own message and value of parameters:

  • client_uniq_country_login_count: Sets maximum number of unique countries to allow user to log in using sasl.
  • client_uniq_ip_login_count: Sets maximum number of unique IP addresses to allow user to log in using sasl.
# Anti spam botnet rule:
#   This example shows how to limit e-mail address defined by `sasl_username`
#   to be able to login from max. 5 different countries or 20 different IP
#   addresses, otherwise it will be blocked from sending messages.

id=BAN_BOTNET_COUNTRY ;
   sasl_username=~^(.+)$ ;
   client_uniq_country_login_count > 5 ;
   action=rate(sasl_username/1/3600/554 Your mail account ($$sasl_username) was compromised. Please change your password immediately after next login.) ;

id=BAN_BOTNET_IP ;
   sasl_username=~^(.+)$ ;
   client_uniq_ip_login_count > 20 ;
   action=rate(sasl_username/1/3600/554 Your mail account ($$sasl_username) was compromised. Please change your password immediately after next login.) ;

Database backend configuration

Update configuration file /etc/postfix/anti-spam.conf with your credentials to selected database backend. Don't forget to use proper driver and port.

In case you use different path such as /etc/postfix/anti-spam.conf and /etc/postfix/anti-spam-sql-st.conf to main configuration file, export environment variables POSTFWD_ANTISPAM_MAIN_CONFIG_PATH and POSTFWD_ANTISPAM_SQL_STATEMENTS_CONFIG_PATH with your custom path.

[database]
# driver = Pg
driver = mysql
database = test
host = localhost
# port = 5432
port = 3306
userid = testuser
password = password

Application configuration

The plugin is by default configured to remove records for users with last login date older than 24 hours. This interval can be changed in configuration app.db_flush_interval.

Plugin looks by default for GeoIP database file in path /usr/local/share/GeoIP/GeoLite2-Country.mmdb. You can override this path in configuration app.geoip_db_path.

You can whitelist set of IP addresses or subnets in CIDR format by using configuration setting app.ip_whitelist. Whitelisting means, that if client logs into email account from IP address, which IS in whitelist, it will NOT increment login count for this pair of sasl_username|client_address.

[app]
# flush database records with last login older than 1 day
db_flush_interval = 86400
geoip_db_path = /usr/local/share/GeoIP/GeoLite2-Country.mmdb
# IP whitelist must be valid comma separated strings in CIDR format without whitespaces.
# it specifies IP addresses which will NOT be counted into user logins database.
ip_whitelist = 198.51.100.0/24,203.0.113.123/32
# ip_whitelist_path = /etc/postfwd/ip_whitelist.txt

Logging

Plugin is by default logging into standard output. This can be changed in configuration file by setting value for logging.logfile.

You can disable logging completely by updating value of statement logging.enable to 0.

[logging]
# enable(1) or disable(0) logging
enable = 1
# remove statement `logfile`, or set it to empty `logfile = ` to log into STDOUT
logfile = /var/log/postfwd_plugin.log
autoflush = 0
# make log after exceeding unique country count limit
country_limit = 5
# make log after exceeding unique ip count limit
ip_limit = 20

If you use logrotate to rotate anti-spam logs, use option copytruncate which prevents logging errors when log file is rotated.

Useful database queries

Plugin stores interesting statistical information in the database. To query those statistics you can use predefined SELECTs located in separate file DB-Queries.md.

Development and testing

Prototyping with Docker

Complete development environment with postfwd, anti-spam plugin and mysql/postgresql database configured together can be run with single command from directory tests/:

  • PostgreSQL: export POSTFWD_ANTISPAM_MAIN_CONFIG_PATH=/etc/postfwd/01-dev-anti-spam-postgres-geoip2.conf; docker-compose -f compose-dev-postgresql.yml up
  • MySQL: export POSTFWD_ANTISPAM_MAIN_CONFIG_PATH=/etc/postfwd/02-dev-anti-spam-mysql-geoip2.conf; docker-compose -f compose-dev-mysql.yml up

Note for overriding postfwd arguments:

  • Most important arguments to run postfwd in Docker are --stdout and --nodaemon. These arguments configure postfwd to log into standard output and stay in foreground.
  • For running postfwd plugin, you also need to set argument --plugins <path-to-plugin> to correct location of plugin.

MaxMind test database tests/GeoLite2-Country-Test.mmdb was downloaded from MaxMind-DB repository.

Running tests

Check for proper linting with perlcritic postfwd-anti-spam.plugin.

Run plugin as mentioned in Prototyping with Docker and send SMTP requests to postfwd policy server, or use testing script to check functionality:

# manually send postfwd request
export CLIENT_ADDRESS='1.2.3.4'
export SASL_USERNAME='[email protected]'
nc 127.0.0.1 10040 < <(envsubst < dev-request)

postfwd-anti-geoip-spam-plugin's People

Contributors

francoispe avatar kirecek avatar kollyma avatar lirt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

postfwd-anti-geoip-spam-plugin's Issues

Update README

  • Move subsection Development and Prototyping with Docker to Development and testing and minify it.
  • Remove part (tested with MySQL/PostgreSQL).
  • Releases v1.30 should be first in list.
  • Update TOC.

Geo::IP object version 1.43 does not match bootstrap parameter 1.45

Hi,
Just configured everything and I can't sent any mail (SMTP 451 error). I checked mail log and I got following error: "FATAL: Geo::IP object version 1.43 does not match bootstrap parameter 1.45 at /usr/lib/perl/5.18/DynaLoader.pm line 207." Any idea what can I do? Or what I did wrong?
perl mismatch

Remove copying postfwd into PATH in Dockerfile

Remove line RUN cp /opt/postfwd/sbin/postfwd1 /opt/postfwd/sbin/postfwd2 /usr/sbin/ from Dockerfile and further refactor when upstream update is released.

Currently this change resides in testing branch of postfwd.

Ip whitelisting

I'm using the plugin with postfwd3 2.02 with the following config

id=COUNTRY_LOGIN_COUNT ; \ sasl_username=~^(.+)$ ; \ client_address=!!(x.x.x.x/x) ; \ incr_client_country_login_count != 0 ; \ action=jump(BAN_BOTNET)

If my understanding of the sample config is correct client_address=!!(x.x.x.x/x) is a way to whitelist IPs, however postfwd runs plugins before the rules so the whitelisted IP gets added to the database and still counts toward client_uniq_country_login_count and client_uniq_ip_login_count

Configure CI

Configure CI:

  • Static code checking - Perl Critic and Tidy.
  • Integration test with Docker + Travis

postfwd "Odd number of elements in hash assignment" warning

Hi - this looks like a fantastic plugin! Many thanks for putting it together.

I'm trying to configure it for my setup (Ubuntu 16.04, postfix 3.1.0, postfwd 1.35) and am seeing the following in my mail logs (the STDIN line numbers vary):

Jun 15 15:12:01 lorina postfwd2/policy[18554]: warning: Odd number of elements in hash assignment at /usr/sbin/postfwd line 1631, <STDIN> line 588.?
Jun 15 15:12:01 lorina postfwd2/policy[18554]: message repeated 2 times: [ warning: Odd number of elements in hash assignment at /usr/sbin/postfwd line 1631, <STDIN> line 588.?]

I also don't see any records being written to the database when sasl users send mail. The debug logs look like this:

2019-06-15 15:03:43 postfwd::anti-spam-plugin INFO[18545]: Logging destination is file '/tmp/postfwd_plugin.log'
2019-06-15 15:03:43 postfwd::anti-spam-plugin INFO[18545]: GeoIP Database Informations: GEO-106FREE 20160408 Bu
2019-06-15 15:03:43 postfwd::anti-spam-plugin INFO[18545]: GeoIP Database Edition: 1
2019-06-15 15:03:43 postfwd::anti-spam-plugin INFO[18545]: Starting postfwd plugin with dsn 'DBI:mysql:database=pf_antispam;host=10.10.10.100;port=3306'
2019-06-15 15:03:43 postfwd::anti-spam-plugin INFO[18545]: Database connection successful
2019-06-15 15:03:43 postfwd::anti-spam-plugin INFO[18545]: Creating table postfwd_logins if it does not exist
2019-06-15 15:03:43 postfwd::anti-spam-plugin INFO[18545]: Table was created successfully and plugin is correctly initialized.
2019-06-15 15:06:16 postfwd::anti-spam-plugin INFO[18604]: Database connection dropped (rc=). Reconnecting to database.
2019-06-15 15:06:19 postfwd::anti-spam-plugin INFO[18554]: Database connection dropped (rc=). Reconnecting to database.
2019-06-15 15:06:20 postfwd::anti-spam-plugin INFO[18557]: Database connection dropped (rc=). Reconnecting to database.
2019-06-15 15:06:24 postfwd::anti-spam-plugin INFO[18556]: Database connection dropped (rc=). Reconnecting to database.
2019-06-15 15:06:26 postfwd::anti-spam-plugin INFO[18578]: Database connection dropped (rc=). Reconnecting to database.
2019-06-15 15:06:32 postfwd::anti-spam-plugin INFO[18550]: Database connection dropped (rc=). Reconnecting to database.
2019-06-15 15:06:32 postfwd::anti-spam-plugin INFO[18558]: Database connection dropped (rc=). Reconnecting to database.

Any tips on what I might be doing wrong?

Also, in the PRIVATE_RANGES config, it has some examples. I have a LAN on 10.10.10.0/24 and a single WAN IP. So do I put the following in that stanza?

&&PRIVATE_RANGES { \
   client_address=!!(10.10.10.0/24) ; \
   client_address=!!(my.wan.ip.xx/32) ; \
};

Drop support for old versions

Drop support for old versions, keep only last from 1.x and 2.x release line.

Also fix some inconsistencies in readme.

Optimize docker image

Docker image contains a lot of not necessary packages. Try to optimize its size to minimal. Using multi-stage build can probably help here.

postfwd version and database

Hello,
I set up plugin v1.50.0
I configure postfix with postfwd : I test data restriction through postfwd, it is working.
I send mails from thunderbird and webmail (nextcloud) but mysql database table remains empty...
I am not sure to understand how it works : should I have an entry in database for all emails sent with a different IP ?

I suspect postfwd not being right version.
I have ubuntu 20.04, run 'apt postfwd'.
My server runs postfwd2, and, I don't see how to upgrade to postfwd3... as per plugin install requirement

Can you help me understand how to switch to postfwd3 ?
Many thanks

PostgreSQL support

Hello, would be possible to add support for PostgreSQL?

It works a little if I modify database connection parameters. But it stops working after quite short time. I think the key problem is that autoreconnect for PostgreSQL is not implemented in Perl.
I see following error in log:
postfwd2/policy[6588]: FATAL: Can't call method "errstr" on an undefined value at /etc/postfix/postfwd-anti-spam.plugin line 172, line 28.?

warning: DBI connect

Hello im supposing that this works with Maria Database as well, if not can i adjust it?
Screenshot_20200530_002549

Allow logging to STDOUT

Allow logging to STDOUT by omitting configuration statement logfile in section [logging] or keeping it empty (logfile = "").

Upgrade GeoIP database to version 2

GeoIP databases version 1 are obsolete and hard to download. Migrate code to GeoIP v2 Perl module and rewrite install script, documentation and Docker to use GeoIP databases v2.

Consider using geoipupdate in Docker.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.