HOWTO Postfix Virtual Mail + IMAP/POP3 + Spamassassin + Roundcube Webmail
From Gentoo Linux Wiki
| It has been suggested that this article be split into multiple articles accessible from a disambiguation or index page. (Discuss) |
This walkthrough assumes you have working versions of PHP, Apache and MySQL.
[edit] Goal
This tutorial aims to have a fully functional MTA with virtual mail, spam filtering, and webmail. With Roundcube as our webmail we can easily move messages in and out of a Junk folder that maildrop and SpamAssassin will sync with. A nightly cron will retrain SpamAssassin based off of what is in the Junk folder and what is not.
Using courier-authlib we can check against the system's shadow file as well as a mysql database for user authentication. Local users will enjoy the convenience of not having to supply their full email addresses for imap/pop3/webmail login.
Note: POP3 clients like to erase all inbox messages from the server. There is no problem with this, but it will raise the learning curve for SpamAssassin since there won't be any legitimate email for contrast.
This walkthrough is constructed in a way that related tasks are lumped together.
[edit] What we're merging and why
Our MTAs, Antivirus, and Spamfilter:
emerge -av postfix courier-imap amavisd-new spamassassin
Pull down maildrop for mail delivery as well as server-wide message rules. This will help sync SpamAssassin and Roundcube
emerge -av maildrop
Configuring php to play nice with Roundcube
echo "dev-lang/php cli imap apache2 mysql" >> /etc/portage/package.use emerge -av dev-lang/php PEAR-PEAR
At this point it would be a good idea to reload Apache and make sure your PHP hasn't broken.
/etc/init.d/apache2 reload
[edit] Add the vmail user
The vmail user will be our go-between for keeping track of virtual users and their mail.
useradd -m -s /bin/false -d /home/vmail vmail passwd vmail
vmail's UID and mail's GID will play a role later on, so get those and write them down.
id vmail
Mail for virtual users will be stored in /home/vmail/virtualdomain.tld/user/mail
To create a maildir for user@virtualdomain.tld
mkdir -p /home/vmail/virtualdomain.tld/user maildirmake /home/vmail/virtualdomain.tld/user/mail chown -R vmail:mail /home/vmail/virtualdomain.tld chmod -R o-rwx /home/vmail/virtualdomain.tld
[edit] Setting up databases
mysql -uroot -p
Aliases and Virtual Email database
create database vmail; use vmail; grant all on *.* to vmail identified by 'password'; create table aliases ( alias varchar(255) not null primary key, crypt varchar(255) not null, destination varchar(255) not null, modified timestamp );
Roundcube's Workspace
create database roundcubemail; use roundcubemail; grant all on *.* to roundcube identified by 'password';
SpamAssassin's Workspace
create database spamassassin; use spamassassin; grant all on *.* to spamassassin identified by 'password'; -- bayes database structure -- taken from http://spamassassin.apache.org/full/3.0.x/dist/sql/bayes_mysql.sql CREATE TABLE bayes_expire ( id int(11) NOT NULL default '0', runtime int(11) NOT NULL default '0', KEY bayes_expire_idx1 (id) ) TYPE=MyISAM; CREATE TABLE bayes_global_vars ( variable varchar(30) NOT NULL default , value varchar(200) NOT NULL default , PRIMARY KEY (variable) ) TYPE=MyISAM; INSERT INTO bayes_global_vars VALUES ('VERSION','3'); CREATE TABLE bayes_seen ( id int(11) NOT NULL default '0', msgid varchar(200) binary NOT NULL default , flag char(1) NOT NULL default , PRIMARY KEY (id,msgid) ) TYPE=MyISAM; CREATE TABLE bayes_token ( id int(11) NOT NULL default '0', token char(5) NOT NULL default , spam_count int(11) NOT NULL default '0', ham_count int(11) NOT NULL default '0', atime int(11) NOT NULL default '0', PRIMARY KEY (id, token) ) TYPE=MyISAM; CREATE TABLE bayes_vars ( id int(11) NOT NULL AUTO_INCREMENT, username varchar(200) NOT NULL default , spam_count int(11) NOT NULL default '0', ham_count int(11) NOT NULL default '0', token_count int(11) NOT NULL default '0', last_expire int(11) NOT NULL default '0', last_atime_delta int(11) NOT NULL default '0', last_expire_reduce int(11) NOT NULL default '0', oldest_token_age int(11) NOT NULL default '2147483647', newest_token_age int(11) NOT NULL default '0', PRIMARY KEY (id), UNIQUE bayes_vars_idx1 (username) ) TYPE=MyISAM;
Verify that all logins work before continuing.
[edit] Notes on the vmail database
In the vmail database a forward is defined by a blank crypt field. You can forward anything any which way. Anything without an @domain on it will be interpreted as a local user. The following configuration is a daisy chain of forwards, starting with a catchall, all leading straight to joe's localhost mailbox. Note: Forwarding to a forward is bad practice! This is merely here to demonstrate a complex configuration!
mysql> select alias,destination from aliases where crypt=""; +---------------------------+---------------------------+ | alias | destination | +---------------------------+---------------------------+ | @virtualdomain.tld | virtual@virtualdomain.tld | | virtual@virtualdomain.tld | virtual@anotherdomain.tld | | virtual@anotherdomain.tld | joe | +---------------------------+---------------------------+
Here are more upstanding examples, keeping forwarding single-leveled.
+-----------------------+------------------+ | alias | destination | +-----------------------+------------------+ | dave | dave@virtual.tld | | bob@virtual.tld | bob@another.tld | | webmaster@virtual.tld | john | +-----------------------+------------------+
Virtual mailbox entries in the vmail database will look something like this.
mysql> select alias,crypt,destination from aliases where crypt!=""; +------------------+------------------------------------+-------------------------------+ | alias | crypt | destination | +------------------+------------------------------------+-------------------------------+ | dave@virtual.tld | $1$1ZAEvOvu$oIrrkJUAE4TPdLK93St571 | /home/vmail/virtual.tld/dave/ | | bob@another.tld | $1$pYB1SOWo$bekEPusqEN0s.llW9PMc5/ | /home/vmail/another.tld/bob/ | +---------------------------------------------------------------------------------------+
- Trailing slashes for destinations are important!
- Note that the crypt is in standard Unix DES-based encryption
- Since john has local delivery authlib will find him in shadow, so there is no need to make a virtual mailbox entry for him.
[edit] Configuring Postfix
Remember, deep breaths.
/etc/postfix/main.cf
# myhostname is the primary FQDN of your mailserver myhostname = example.com mydomain = $myhostname mydestination = $myhostname, localhost mynetworks_style = host # allow mail for domains that are not $myhostname virtual_alias_domains = virtualdomain.tld, anothervirtualdomain.tld, etc virtual_mailbox_domains = $virtual_alias_domains #alias_maps = ... # set up delivery virtual_alias_maps = mysql:/etc/postfix/mysql-aliases.cf virtual_mailbox_maps = mysql:/etc/postfix/mysql-mailboxes.cf local_recipient_maps = mailbox_command = /usr/bin/maildrop -d "$USER" -f "$SENDER" "$EXTENSION" fallback_transport = maildrop virtual_transport = maildrop home_mailbox = mail/
Add content-filter details at the bottom
content_filter = smtp-amavis:[127.0.0.1]:10025 biff = no message_size_limit = 25000000 smtpd_helo_required = yes # uncomment these (and remove the linebreaks) to be more strict #smtpd_helo_restrictions = permit_mynetworks, reject_unauth_pipelining, reject_non_fqdn_hostname, # reject_unknown_hostname, reject_invalid_hostname, permit #smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, # reject_multi_recipient_bounce, reject_non_fqdn_recipient, # reject_unknown_recipient_domain
/etc/postfix/mysql-mailboxes.cf
# determine all real mail addresses # this prevents catchalls from catching legitimate mail user = vmail password = password dbname = vmail table = aliases select_field = alias where_field = alias hosts = 127.0.0.1
/etc/postfix/mysql-aliases.cf
# determine forwards and catchalls user = vmail password = password dbname = vmail table = aliases select_field = destination where_field = alias additional_conditions = and crypt="" hosts = 127.0.0.1
/etc/postfix/master.cf
maildrop unix - n n - - pipe
flags=DRhu user=vmail argv=/usr/bin/maildrop -d ${recipient}
Add amavis details at the bottom
smtp-amavis unix - - y - 4 smtp -o smtp_data_done_timeout=1200 -o disable_dns_lookups=yes 127.0.0.1:10025 inet n - y - - smtpd -o content_filter= -o local_recipient_maps= -o relay_recipient_maps= -o smtpd_restriction_classes= -o smtpd_helo_restrictions= -o smtpd_sender_restrictions= -o smtpd_recipient_restrictions=permit_mynetworks,reject -o mynetworks=127.0.0.0/8 -o strict_rfc821_envelopes=yes
[edit] Configuring SpamAssassin and Amavis
/etc/conf.d/spamd
SPAMD_OPTS="-m 5 -u vmail"
/etc/spamassassin/local.cf
use_bayes 1 bayes_auto_learn 1 bayes_auto_learn_threshold_nonspam -1 bayes_auto_learn_threshold_spam 14.00 bayes_ignore_header X-Bogosity bayes_ignore_header X-Spam-Flag bayes_ignore_header X-Spam-Status
/etc/spamassassin/secrets.cf
#(Tell Spamassissin to use MySQL for bayes data bayes_store_module Mail::SpamAssassin::BayesStore::SQL bayes_sql_dsn DBI:mysql:spamassassin:localhost:3306 bayes_sql_username spamassassin bayes_sql_password password
/etc/amavisd.conf
$mydomain = 'example.com'; #fdqn of this server $myhostame = 'example.com'; #fqdn of this server
[edit] Configuring maildrop
Maildrop is going to take any email marked as spam by spamassassin and place it into a Junk folder alongside Inbox etc. Roundcube will automatically pick up on this.
/etc/maildroprc
DEFAULT = "$HOME/mail" MAILDIR = $DEFAULT
Add to the bottom of maildroprc
#Drop spam in junk folder
if (/^X-Spam-Status: Yes/)
{
exception {
to $DEFAULT/.Junk/
}
}
[edit] Configuring Authlib
Maildrop and IMAP/POP3 authenticate using whatever authlib specifies. Here we will specify to check our virtual users and fall back on local users.
/etc/courier/authlib/authdaemonrc
##NAME: authmodulelist:2 #Email boxes can belong to virtual and actual users. authmodulelist="authmysql authshadow"
/etc/courier/authlib/authmysqlrc
Connection details
MYSQL_SERVER localhost MYSQL_USERNAME vmail MYSQL_PASSWORD password
Database details (insert vmail's UID and mail's GID for these values, single quoted.
MYSQL_DATABASE vmail MYSQL_USER_TABLE aliases MYSQL_CRYPT_PWFIELD crypt #DEFAULT_DOMAIN ... MYSQL_UID_FIELD 'vmail's UID' MYSQL_GID_FIELD 'mail's GID' MYSQL_LOGIN_FIELD alias MYSQL_HOME_FIELD destination #MYSQL_NAME_FIELD ...
[edit] Configuring IMAP/POP3
/etc/courier-imap/imapd
Listen on all interfaces.
ADDRESS=0
Don't purge trash.
IMAP_EMPTYTRASH=
Mail is stored here:
MAILDIR=mail MAILDIRPATH=mail
/etc/courier-imap/pop3d
Mail is stored here:
MAILDIR=mail MAILDIRPATH=mail
[edit] Roundcube Webmail
Install Roundcube to /home/vmail/public_html
See Roundcube
[edit] Create webmail subdomains
Edit apache's vhosts file and add at the bottom:
# all mail <VirtualHost *:80> ServerName webmail.somedomain.tld ServerAlias webmail.anotherdomain.tld ServerAlias webmail.andanotherdomain.tld #ServerAlias webmail.etc.etc ServerPath /usr/lib/apache2 DocumentRoot /home/vmail/public_html </VirtualHost>
Make sure the DNS for these domains have a 'webmail' CNAME set to your mail server.
[edit] Start your engines
rc-update add apache2 default rc-update add courier-authlib default rc-update add courier-imapd default rc-update add amavisd-new default rc-update add spamd default rc-update add postfix default rc-update add clamd default
/etc/init.d/apache2 reload /etc/init.d/courier-authlib start /etc/init.d/courier-imapd start /etc/init.d/amavisd-new start /etc/init.d/spamd start /etc/init.d/postfix start /etc/init.d/clamd start
If all services started without problems it is time to start testing. If you haven't already, it will make things easier for you if you use phpMyAdmin to administer the vmail database: phpMyAdmin uses MySQL to generate crypts, which are not the same as those found in shadow. A workaround is to generate one via command-line and then copy it.
php -r 'echo crypt("password")."\n";'
Open a couple terminals, one for tweaking settings if needed and one for monitoring the mail log. We will want to monitor the mail log's most recent entries:
less +F /var/log/mail.log
and
less +F /var/log/messages
Add some forwards and virtual mailboxes and send them some mail from external email addresses. From this point you can actively watch whats happening inside the server as it routes mail.
Visit your webmail locations to make sure they are all pointed correctly and that you can log in. Try sending mail between virtual and local users to make sure everything is working properly.
[edit] Routine maintenance
When adding new virtual domains you must update both /etc/postfix/main.cf and /etc/amavisd.conf. A nightly cron can be added to update clamav's virus definitions and train SpamAssassin.
~/spamlearn.sh
echo "Learning from local users:"; for user in $(ls -1 /home); do if [ -d /home/$user/mail/.Junk ]; then echo " - $user's spam" echo -n " - " sa-learn --spam /home/$user/mail/.Junk/cur -u $user echo " - $user's ham" echo -n " - " sa-learn --ham /home/$user/mail/cur -u $user fi done echo echo "Learning from virtual users:"; for domain in $(ls -1 /home/vmail); do for user in $(ls -1 /home/vmail/$domain); do if [ -d /home/vmail/$domain/$user/mail/.Junk ]; then echo " - $user@$domain's spam" echo -n " - " sa-learn --spam /home/vmail/$domain/$user/mail/.Junk/cur -u $user@$domain echo " - $user@$domain's ham" echo -n " - " sa-learn --ham /home/vmail/$domain/$user/mail/cur -u $user@$domain fi done done echo echo "Done."
chmod u+x ~/spamlearn.sh
crontab -e
# spamassassin learning every night at midnight 0 0 * * * ~/spamlearn.sh
# update clamav's virus definitions nightly 0 0 * * * freshclam
