HOWTO Spam Filtering with Spamdyke in front of Qmail

From Gentoo Linux Wiki

Jump to: navigation, search

Contents

[edit] Intro

This HOWTO should give an overview on using spamdyke SMTP filter in front of the mail-mta/netqmail mailserver. From the spamdyke website:

spamdyke is a filter for monitoring and intercepting SMTP connections between a remote host and a qmail server. Spam is blocked while the remote server (spammer) is still connected; no additional processing or storage is needed. In addition to all of its anti-spam filters, spamdyke also includes a number of features to enhance qmail.

This HOWTO asumes a working mail-mta/netqmail installation with optional net-mail/vpopmail support. If you don't have these pre-conditions fulfilled, please consult the official netqmail/vpopmail Virtual Mail Hosting System Guide or one or more of the following resources to get your environment ready:

[edit] Pre-conditions

This HOWTO asumes a running mail-mta/netqmail installation, optionally enhanced by net-mail/vpopmail. Other components that can typically be found in such environments, like mail-filter/maildrop and net-mail/qmailadmin, should not be affected.

This howto was created on a "i686-pc-linux-gnu" with the following ebuilds installed:

Different environments may affect the way this HOWTO should work and can lead to strange effects. If you experience anything in this direction, please leave a note here or on the discussion page.

Warning: Versions of spamdyke around 2.6.3 did not work with net-mail/vpopmail without clearpasswd useflag, because of a bug (see discussion page). This is solved in the most recent version, so you do not necessarily need to apply clearpasswd or migrate to it.

[edit] Installation

You can install spamdyke from source or emerge it from Portage.

[edit] From Portage

If you need/use SMTP-AUTH with your Qmail installation, add the following to /etc/portage/package.use:

File: /etc/portage/package.use 
mail-filter/spamdyke tls

If the package is masked, add the following to /etc/portage/package.keywords:

File: /etc/portage/package.keywords 
=mail-filter/spamdyke-3.1.8

Then emerge the package:

Code: Emerging spamdyke 
 # emerge -av spamdyke

The binary is installed to /usr/bin whereas the configuration is in /etc/spamdyke/spamdyke.conf

[edit] From Source

This HOWTO is using version 3.1.6 of spamdyke. If you find a newer version of the 3.1.x series the following steps should work fine, too.

[edit] Download and unpack

Download spamdyke-3.1.6 from the spamdyke releases website to a directory of your choice and upack the tar ball:

wget http://www.spamdyke.org/releases/spamdyke-3.1.6.tgz

tar -xvzpf spamdyke-3.1.6.tgz

[edit] Compile

spamdyke does only require the usual compilation procedure in the sub-folder spamdyke/ of the just extracted spamdyke-3.1.6/ directory.

cd spamdyke-3.1.6/spamdyke

./configure

make

The ./configure script automatically detects if you need TLS support (by checking for OpenSSL) and compiles spamdyke accordingly. The resulting spamdyke binary is the only file produced and needed.


Tip: To check if TLS support has been correctly compiled into your spamdyke binary just run it with the -v option. The version should contain '+TLS'.

[edit] Install

To install the spamdyke binary to your system copy it over to /usr/local/bin/spamdyke.

cp spamdyke /usr/local/bin/spamdyke

[edit] Configuration

After successfully installing the spamdyke binary, it needs to be integrated into the mail-mta/netqmail setup. The documentation spamdyke-3.1.6/documentation/INSTALL.txt suggests to configure spamdyke directly in /service/qmail-smtpd/run, but to stick with Gentoo style, changes are only performed in /var/qmail/control/conf-smtpd.

[edit] Getting started

spamdyke needs to be placed before qmail-smtpd receives the incoming mail. It will then decide upon its configuration if an incoming mail should be directly rejected or if it should be passed on to this MTA. To achieve this on Gentoo, the QMAIL_SMTP_PRE variable is the corrects place to hook spamdyke in:

File: /var/qmail/control/conf-smtpd 
SPAMDYKE_OPTIONS="--hostname '$HOSTNAME' --config-file /etc/spamdyke/spamdyke.conf"
QMAIL_SMTP_PRE="${QMAIL_SMTP_PRE} /usr/local/bin/spamdyke $SPAMDYKE_OPTIONS"


This adds spamdyke right before Qmail accepts an incoming connection. The filter needs to know the hostname of the local machine which is available through the environment variable $HOSTNAME, if you configured it correctly in /etc/conf.d/hostname.

The second parameter is a config file that contains all further configurations. spamdyke can also get all its configuration values from commandline options, but maintaining them in a config file is a) more comfortable and b) it is even faster, as the spamdyke website states:

Note: It may seem that scanning a configuration file instead of the command line would impose a performance penalty each time spamdyke is started. However, the reverse seems to be true. Some rudimentary testing has indicated the configuration files are actually faster.

[edit] Basic configuration

The /etc/spamdyke/spamdyke.conf should contain the following lines for testing:

File: /etc/spamdyke/spamdyke.conf 
log-level=2
local-domains-file=/var/qmail/control/rcpthosts
tls-certificate-file=/var/qmail/control/servercert.pem

Log level 2 indicates that all errors and info messages should be logged. You should switch that to level 1 after testing, because spamdyke will probably log a loooot of blocked spam. ;) The local-domains setting is needed since spamdyke will reject all emails to other domains automatically, of no SMTP-Auth is provided. The last setting is only neccessary if you use TLS/SSL to secure connections and want spamdyke to inspect such connections, too.

[edit] Getting spamdyke to run

To make Qmail use your fresh installed spamdyke, just restart Qmail:

/etc/init.d/svscan restart

Note: The qmail services sometimes take a longer time to stop, if communication with a remote host takes long. You might need to stop svscan, kill pending processes manually and then start it again.

[edit] Options

spamdyke supports a huge lot of options to affect its behaviour. The most important ones for fighting spam are blacklisting options, that allow you to define which mails should be rejected by spamdyke. You get a complete overview on available options by calling spamdyke -h. These are also explained in detail in spamdyke-3.1.6/documentation/README.txt.

In the configuration file you can use every long option without the leading 2 dashes. If a value is expected, just add a = and the value after that.

Recommended options by this article are:

File: /etc/spamdyke/spamdyke.conf 
# Reject the connection if no rDNS name exists for the remote servers IP
reject-empty-rdns

# Reject the connection of no rDNS name could be resolved for the remote servers IP
reject-unresolvable-rdns

#Checks the senders domain name for MX or A records and rejects the connection if none is found.
reject-missing-sender-mx

With these options spamdyke will reject most of the connections from dialup networks, since those usually don't have a reverse DNS entry setup. Beside that, email that appears to come from a domain that does not have an MX or even an A DNS entry, will be rejected, which is often the case for spammer domains. This will most probably reduce already a huge lot of your spam, sent by Windoze PCs infected with bots and the like.

Warning: If you retreive emails from email servers that reside on dialup hosts, you cannot use these options. However, in most cases you don't have such mail servers.

[edit] SSL / TLS

If your mail-mta/netqmail is compiled with ssl support, spamdyke cannot examine emails send through encrypted connections and can only include the sender IP in its checks. To enable spamdyke to examine the email content (especially the sender and receiver) it needs to use the same SSL certificate than qmail does:

File: /etc/spamdyke/spamdyke.conf 
# Makes spamdyke use your Qmails server certificate to examine secured connections.
tls-certificate-file=/var/qmail/control/servercert.pem

[edit] SMTP-Auth

Spamdyke does now support SMTP-Auth out of the box. It makes Qmail serve the authentication and examines the status. If authentication fails, spamdyke will also reject the connection.

[edit] Blacklisting

The setup this HOWTO is based on makes use of 2 differnt kinds of blacklists:

ip-blacklist-file 
Allows you to specify a file which contains a list of IP addresses to block.
check-dnsrbl 
Although a DNSBL lookup is slower than local file updates I'm using several DNSBLs in addition to the local IP based one.
File: /etc/spamdyke/spamdyke.conf 
ip-blacklist-file=/var/qmail/spamdyke/nixspam.ipbl
check-dnsrbl=my.dnsbl.server.example.com
Note: I'm using the NiX spam black list here, which is provided by the German iX magazine, and download this one every hour. This bans already a large number of spam send to German boxes and blacklist lookups in a local file are cheap. See the tips and tricks section of this article for how to update the NiX spam black list.

[edit] Whitelisting

Whitelisting is the opposite of blacklisting. For all hosts that are found in an IP whitelist all further checks are ignored and email is just forwarded to Qmail. Therefore you should only use this features with hosts for that you can definitly ensure they are not compromised or send spam because of another reason.

ip-whitelist-file 
The given file contains a newline seperated list of IP addresses.
File: /etc/spamdyke/spamdyke.conf 
ip-whitelist-file=/var/qmail/spamdyke/dotxp.ipwl

[edit] Tips and tricks

This section provides practical tips and tricks, which are not neccessarily useful for everyone. It is recommended to just read through it and check what you find interesting or to find creative ideas.

[edit] NiX spam

To update the NiX-spam blacklist, run the following script every 30 minutes via a CRON job:

File: update-nix-spam.sh 
#!/bin/bash
FILE="/var/spamdyke/blacklists/nixspam.ipbl"
URL="http://www.heise.de/ix/nixspam/nixspam.blackmatches"
USER="qmaild"
/usr/bin/lynx -source $URL  | grep -v '^#' | awk '{ print $2; }' > "$FILE"
/bin/chown "$USER" "$FILE"

NiX spam makes only sense if you are located in Germany or at least western Europe. If you have other IP based blacklists to use, this script should be adjustable to your needs. Remember to create the directory where blacklists should be stored and to make them readble for the user running spamdyke.

[edit] DNSBL

For DNSBL I use the following providers:

  • zen.spamhaus.org
  • list.dsbl.org (Status as of 13.07.2008: offline)
  • cbl.abuseat.org (already implemented in the Spamhaus database)
  • spamsources.fabel.dk
  • ix.dnsbl.manitu.net

The last one is the DNSBL server of NiX-spam, which contains more and especially more up2date IP addresses, which might find spammers even if their IP address is not in the local IPBL.

[edit] Further reading

Interesting links related to this HOWTO:

Retrieved from "http://www.gentoo-wiki.com/HOWTO_Spam_Filtering_with_Spamdyke_in_front_of_Qmail"
Personal tools