[edit]

Help hurricane victims by donating to the American Red Cross.

Qmail-SpamAssassin

From Ollie's Web Site

Contents

[edit]

qmail-spamassassin

[edit]

Project

To set up a qmail/spamassassin front end for exchange. The general concept is: 

internet-->qmail_server-->exchange_server

where qmail_server is public-smtp-accessible through the firewall, while the exchange_server is only acessible from the qmail_server. The qmail code is modified to scan all inbound (or in and out, depending on configuration) emails through virus and spam filters.

The virus filter quarantines the virus, while the spam filter adds headers (or modifies Subject: if exchange/outlook is unable to filter on headers) for identified spam.

[edit]

Operating Notes: sulfurous

Here are detailed operating notes for the server: SulfurousQmail

[edit]

Install qmail

  • qmail.org lives here.
  • Follow Life with Qmail, an excellent installation guide.
  • Here is another How-to that includes use of many additional patches.
  • Start with netqmail, which is qmail plus several crucial patches. This gives installation instructions. Section 2.5.2 talks about this, but here is my version:
  1. download netqmail
  2. unpack it to someplace like /usr/local/src. This should give you a new folder called /usr/local/src/netqmail-1.05.
  3. follow the directions in netqmail-1.05/README. This unpacks the clean qmail archive into a new folder netqmail-1.05/netqmail-1.05 (confusing, huh?) with the patch already applied.
  4. continue with the LWQ 2.5.2 instructions for compiling qmail inside the modified source directory, /usr/local/src/netqmail-1.05/netqmail-1.05.

For testing, I suggest:

  1. set up mx record for mailqm.hamonusa.com pointing at new qmail_server.
  2. initially set up a couple of real users on the qmail_server box to confirm the basic qmail delivery mechanism is working properly.
  3. Do all the recommended qmail tests.
[edit]

Forward all emails in qmail

:[ip_addr_of_exchg_server] (note the brackets around ip address)

Here's what I did to get hamonusaqm.com to forward to acheson.org:

  1. Loaded "hamonusaqm.com" into rcpthosts.
  2. Did not load into locals.
  3. Put "hamonusaqm.com:[64.81.195.127]" into smtproutes.
  4. Restarted qmail.

Note: on the other end I had to add "hamonusaqm.com" to both locals and rcpthosts.

The next step was to forward to the exchange server. The only change needed was to change the ip address in smptroutes.

8/28 - hmm. Seems it is hard to have a common user list for two different domains be served by one exchange server. Michael fixed for a couple of test users.

Testing here involves checking the forwarding mechanism. Try sending emails to oacheson@mailqm.hamonusa.com to see if it forwards properly to oacheson in the exchange server. Try repeatedly with other users and with users who do not exist.

[edit]

Set up virus and spam filters

[edit]

SpamAssassin

spamassassin

See http://wiki.apache.org/spamassassin/IntegratedInQmailWithQmailScanner and http://wiki.apache.org/spamassassin/UsingSiteWide for system-wide install.

Webmail server guide

Here is an article about feeding spam and ham back to sa from an exchange server.

Ultimately used debian testing package. Had to add user spamd group spamd, and enable in /etc/default/spamassassin.

In /etc/default/spamassassin make sure have "-u spamd " in options line.

Debian/spamassassin howto. Note: this covered:

  • Setting up Pyzor, an open source version of Razor. See also Steps to Block Spam with Pyzor and UsingPyzor on Spamassassin wiki.
    • /home/spamd/.pyzor/ -- check permissions. The subdirectory must be owned by user and group spamd, which must have RWX access (chmod 700 /home/spamd/.pyzor).
    • then run "su spamd -c 'pyzor discover'" to generate the server list.
    • enable in /etc/spamassassin/v310.pre
    • restart spamassassin: /etc/init.d/spamassassin restart
    • spamassassin --lint -D indicates was able to contact pyzor server.
  • set up razor. See also UsingRazor and RazorSiteWide, both on the Spamassassin wiki.
    • su spamd -c 'razor-client'
    • su spamd -c 'razor-admin --create'
    • su spamd -c 'razor-admin --register'
    • These three set up the directories and files in /home/spamd/.razor and register with razor.
    • enable in /etc/spamassassin/v310.pre
    • restart spamassassin: /etc/init.d/spamassassin restart
    • su spamd -c 'spamassassin --lint -D' confirmed all working.

Tested using provided test spam.

Note: spamd takes its configuration from many places, including:

  1. /etc/default/spamassassin - some basics, like max-children
  2. /etc/spamassassin/ - contains lots of config files, all of which are included.

Use "spamassassin --lint -D config" to see what config files spamassassin is actually using.

Configuration FAQs at:

[edit]

ClamAV

clamav

Held off installing for now.

[edit]

rblsmtpd

1/19/07 Changed bottom of /var/qmail/supervise/smtpd/run to: 

# lwq mod for rblsmtpd - owa 1/19/07
exec /usr/local/bin/softlimit -m 22000000 \
   /usr/local/bin/tcpserver -v -r -H -l 0  -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" \
       -u "$QMAILDUID" -g "$NOFILESGID" 0 smtp \
       /usr/local/bin/rblsmtpd -b \
       -r list.dsbl.org \
       -r bl.spamcop.net \
       -r sbl.spamhaus.org \
       -r opm.blitzed.org \
       -r cbl.abuseat.org \
       -r dnsbl.njabl.org \
       /var/qmail/bin/qmail-smtpd 2>&1

Need to monitor closely, since when first tried, one of the entries (relay.ordb.org) was dead which caused huge throughput problems. Smtpd queue filled up, sessions timed out and messages weren't getting through.

Later (1/20/07), edited the above to focus in on recommended sites from qmail newsgroup: 

# lwq mod for rblsmtpd - owa 1/19/07 rev 1/20/07
exec /usr/local/bin/softlimit -m 22000000 \
   /usr/local/bin/tcpserver -v -r -H -l 0  -x /etc/tcp.smtp.cdb -c "$MAXSMTPD" \
       -u "$QMAILDUID" -g "$NOFILESGID" 0 smtp \
       /usr/local/bin/rblsmtpd -b \
       -r bl.spamcop.net \
       -r zen.spamhaus.org \
       -r cbl.abuseat.org \
       -r combined.njabl.org \
       -r list.dsbl.org \
       /var/qmail/bin/qmail-smtpd 2>&1
[edit]

Valid recipient checking

We have a huge number of invalid recipients, which contribute to the gross overload of spamassassin. Since we are a gateway for an exchange server, we need to implement some kind of valid rcptto checking.

[edit]

rcpt to checking

Here are a bunch of rcpt to checking patches and routines.

I started with goodrctto-12.patch:

  1. I moved my old source tree to netqmail-1.0.5-BASE in case things didn't work out.
  2. Ran collate.sh.
  3. Did patch per patch instructions.
  4. Stopped qmail (qmailctl stop). Note, you may have to wait a while before qmail-send stops.
  5. Did make setup check, per lwq.
  6. Set up two wildcard domain entries in control/goodrcptto:
    1. @hamonusa.com
    2. @hamonusaqm.com
  7. restarted qmail
  8. Sent test emails to me at both domains. Worked.
  9. Removed entry for @hamonusaqm.com from control/goodrcptto. Sent test email to me at that domain. Immediately bounced.
  10. Added entry for one of my names at hamonusaqm.com. Test email was accepted by sulfurous and passed on to exchange.
  11. Learned how to use control/moregoodrcptto and control/morercptto.cdb
    1. put text list of good fully-qualified email addresses into moregoorcptto (WARNING: make sure it's in unix format, not DOS, by running through script:
      1. tr -d '\15\32' < winfile.txt > unixfile.txt
    2. convert to all lowercase:
      1. cat file1 | tr '[A-Z]' '[a-z]' > file2 (substitute your own files).
    3. run /var/qmail/bin/qmail-newmgrt. This reads control/moregoodrcptto and writes control/moregoodrcptto.cdb.
    4. I removed the wildcard domain entry from control/goodrcptto and immediately qmail started rejecting bad rcptto's.
[edit]

cdb stuff

djb's cdb page.

[edit]

qmail-scanner

qmail-scanner is what knits all the scanning activities together. It inserts itself in front of qmail-queue, allowing it to process emails through virus scanners and spam filters, e.g., clamav and spamassassin.

[edit]

Installation

Get it here: http://qmail-scanner.sourceforge.net/

Important: use both the install doc and the FAQ.php, both of which include essential information. One gotcha! is the need to create qscand user and group.

On installation, it will detect clamav and spamassassin and build them into its code. Remember to use the following options: 

./configure --scanners verbose_spamassassin --install

This is more fully described in the FAQ.

[edit]

tcpserver setup

Additionally, change /etc/tcp.smtp (the rules file for tcpserver) to: 

# modified for qmail-scanner - owa 11/23/04
# no qmail-scanner for mail from 127.0.0.1
127.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-queue"
# use qmail-scanner without spamassassin on mail from local network
# (triggers sa via presence of RELAYCLIENT var)
192.168.1.:allow,RELAYCLIENT="",RBLSMTPD="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue"
# note using c-wrapper not .pl version of qmail-scanner-queue
67.10.194.20:deny
# use qmail-scanner with sa on any mail from the rest of world
:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue"

This uses old qmail-queue for mail from localhost, and new one for all else.

See man tcprules and man tcpserver for more information.

[edit]

Testing

Testing involves sending clean and spam-containing emails through the process and confirming the header rewrite process is occurring. contrib/test_installation.sh worked fine. Handled spam from outside correctly.

[edit]

Timeout

Note: I found several log entries that showed SA (?/?) as the result. I found two causes:

  • Too large message.
  • Timeout of spamc process. This is controlled in qmail-scanner-queue.pl where: 
my $spamc_options=' -t 30 -f';

The '-t 30' sets timeout to 30 sec. I changed to '-t 60' but need to remember that this will be overwritten on the next update of qmail-scanner-queue.

[edit]

Cron routines

I set up an every-other-day routine in cron to roll the logs (in /var/spool/qscan/) and to run /var/qmail/bin/qmail-scanner-queue -z to purge stranded mails. Note the use of the C-wrapper, not the perl script, to avoid the setuidgid problem. I forgot this at first, which led to wrong permissions errors after running cron.

[edit]

Turning scanning on/off

I set up two scripts to quickly turn scanning on and off:

  • /root/tcp.smtp-noscan - kills scanning by (1) updating tcp.smtp, (2) qmailctl cdb, (3) qmailctl restart and (4) killing spamassassin.
  • /root/tcp.smtp-scan - starts scanning by (1) starting spamassassin, (2) updating tcp.smtp, (3) qmailctl cdb, (4) qmailctl restart.
[edit]

qmail-scanner load problems

Well, strictly speaking, I'm having spamassassin problems. qmail-scanner + spamassassin quickly throws off a bunch of processes which then so bog the machine down that it can't keep up. Steps taken:

  1. turned off pyzor, razor, SPF. - no help. Turned back on.
  2. This message indicates SA 3.x is the problem. His system got killed with SA 3.0 at 2,500/hr., a level he had no trouble with at 2.64.
  3. The biggest improvement came from implementing goodrcptto checking. This eliminated a large part of the load. On 9/18, SA kept up until evening, when it bogged down. All of memory and half of swap was in use. I killed the scanning and qmail quickly caught up.
  4. Added memory to 512Mb. Great improvement.
  5. spamassassin max-children - set in /etc/default/spamassassin - now set at 6.
  6. qmail-smtpd - max connections set in /var/qmail/control/concurrencyincoming - now set at 10.
[edit]

qmail-analog

First, expand qmail's logs to more (1 Mb x 20 files for qmail-send) by following what I did on kukla.

Install qmail-analog source. Patch with errno patch in netqmail-1.05.

Install tai64nfrac source. This program converts tai64 timestamp in qmail's multilog files to "seconds since epoch" format qmail-analog expects.

Steps:

  • Convert multilog file to format for qmail-analog: 
tai64nfrac <qmaillogfile >out1
  • Preprocess for qmail-analog: 
/usr/local/qmailanalog/bin/matchup <out1 >out2
  • use various qmail-analog x and z scripts. Here is an overall summary: 
/usr/local/qmailanalog/bin/zoverall <out2

I wrote a script to do all the above and make selecting all of djb's scripts a little easier. It's in the path and is called qm-analog-report. Here is a copy.

[edit]

qmqtool

Handy utility for working on qmail queue.

[edit]

Eliminate Unneeded Processes

It's important to eliminate unneeded processes, both for performance and security. I stopped the following:

  • exim4
  • mysql
  • mysql-ndb
  • mysql-ndb-mgm
  • postgresql-7.4
  • apache2
  • samba
  • nfs-common
  • nfs-kernel-server
  • inetd
  • winbindd

Further, I eliminated the symlinks related to these from the /etc/rc.d/rcN directories using 

update-rc.d -f NAME remove

The -f removes the symlinks while preserving the script in /etc/init.d. Name is the name to be removed.

[edit]

Shorewall

Finally, I installed and configured Shorewall, a stateful firewall process. See here for more information about the install. I closed all ports except 25 (smtp) and 22 (ssh).

[edit]

Troubleshooting

Here's an interesting script that shows how to record smtp dialog selectively: 

Date: Wed, 24 Jan 2007 12:45:25 -0600
From: Charles Cazabon <qmail@discworld.dyndns.org>
Subject: Re: OT: Google Groups: Problems with Email Delivery
To: qmail@list.cr.yp.to
Mail-Followup-To: qmail@list.cr.yp.to

Richard Feldmann <rhun@oaksage.dyndns.org> wrote:
> >
> > Somebody on this list gave me a script to use in my smtpd-run file,
> > which would call qmail-smtpd, selectively invoking recordio depending
> > on an environment variable.
>
> Can that script be shared, Matt?  :)

It's a well-known technique.  You create a shellscript something like:

  #!/bin/sh
  exec ${RECORDIO} /var/qmail/bin/qmail-smtpd

Call it "qmail-smtpd-wrapper.sh".  Then in your /service/smtpd/run script, you
invoke that wrapper instead of qmail-smtpd directly.

When RECORDIO isn't explicitly set (by the tcpserver rules database, etc),
that variable expands to nothing and qmail-smtpd is invoked.  But you can do:

  1.2.3.4:allow,RECORDIO="/path/to/recordio"

... in your rules database to capture SMTP traffic from 1.2.3.4.

Charles
--
--------------------------------------------------------------------------
Charles Cazabon                               <qmail@discworld.dyndns.org>
Read http://pyropus.ca/personal/writings/12-steps-to-qmail-list-bliss.html
My services include qmail consulting.  See http://pyropus.ca/ for details.
--------------------------------------------------------------------------
~
~
[edit]

Administration of the qmail server

Go here for sysadmin items relating to administering sulfurous.

Retrieved from "http://home.acheson.org/index.php/Qmail-SpamAssassin"