|
Quarantine and recover spam message for SpamAssassin, using procmail. |  |  
  
 |
SpamAssassin if very powerfull
for taging possible spam messages, but what should be done with such a message
is left to the installer choice.
To answer the demand of a number of my users, I set-up a way to quaratine the
spam messages, while offering them a way to automatically recover any message
they consider could be a false positive (amont the first moth of testing, less
than 1 of false positive was reported).
This is done through a four steps method, described bellow. The method has
been written and tested for the purpose of CSIM system only, so you can use
it but at your own risks.
This procedure will nicely interface with AMaViS
quarantine system too, allowing users to recover any message that could have
been tagged as containing a virus, but would still have some important information
in the message body.
Step one, receive and quarantine
Quarantine is implemented by procmail, in a format very similar to the interfacing
with procmail as described in SpamAssassin distribution.
| |
<![CDATA[
:0fwE
| /usr/local/bin/spamc -u $LOGNAME
:0e
{
EXITCODE=$?
}
DATE=`date +%Y%m%d%H%M`
PID=`echo $$`
SPAMFILE=$DEFAULT-spam.$DATE.$PID
:0:
* ^X-Spam-Status: Yes
$SPAMFILE]]>
| |
Example of procmailrc file to quarantine incoming spam
Incoming spam is ketp in one file per message, in /var/mail, the filename
is of the format user-spam.date.pid. Where user
is the user name of the recipient and date and pid
are used to compose a unique file name.
There may be a better way to create the date and pid
, but that was inspired from procmail documentation project.
Step two, generate a summary
Following Perl script is used to generate a summary of quarantined spam messages.
This script has been modified:
- to correct a mistake in browsing the spam directory
- to sort the repport by level of spam-iness
| |
<![CDATA[
#!/usr/local/bin/perl
chdir "/var/mail";
$interval=60*60;
$today=time();
while (@user=getpwent) { # for HUMAN users, with group ID in the range [20-99]
next if $user[3] <20 and $user [3]>=100;
$user=$user[0];
opendir DIR, ".";
undef $body;
undef %spam;
undef %from;
undef %date;
undef %subject;
while ($file=readdir DIR) { # look for files with the name user-spam.date.pid
next unless -f $file;
next unless $file=~/^$user-spam\.\d+\.\d+$/;
($a, $b, $c, $d, $e, $f, $g, $h, $i, $mtime, $r)=stat $file;
next unless $today-$mtime>$interval;
# check that the file is at least 60 minutes old
# this add a 60 minutes delay for a file
# to be summarized but should remove every problem
# of locking
undef $subject;
undef $date;
undef $from;
undef $spam;
open IN, "/usr/local/bin/formail -bYczf -X From: -X Subject: -X Date: \
-X X-Spam-Status: < $file |";
# extract the From:, Date:, Spam-status: and Subject: of the
# quarantined mail
while (<IN>) {
$subject=$_ if /^Subject: /;
$date=$_ if /^Date: /;
$from=$_ if /^From: /;
$spam=$_ if /^X-Spam-Status: /;
}
# I keep the information in hash array for further sorting
$subject{$file}=$subject if defined $subject;
$date{$file}=$date if defined $date;
$from{$file}=$from if defined $from;
if ($spam=~/^X-Spam-Status: Yes, (score|hits)=(\d+(\.\d+)?) .*$/) {
# extract the spam level
$spam{$file}=$2;
}
# move the file out of the way
system "/bin/mv -f $file /var/spamassassin/spam/$file";
}
closedir DIR;
undef $i;
foreach $key (sort { $spam{$a} <=> $spam{$b} } (keys %spam)) {
# generate the sorted report
$i++;
$body.=sprintf("%3.0d ", $i);
$body.="File: $key\n";
$body.=" Spam Level: $spam{$key}\n" if defined $spam{$key};
$body.=" $date{$key}" if defined $date{$key};
$body.=" $from{$key}" if defined $from{$key};
$body.=" $subject{$key}" if defined $subject{$key};
$body.="\n";
}
if (defined $body) {
# add the begining of the report and send the email
$body="You have received email(s) that is suspicious spam and was quarantined.
Quarantined messages are kept for 30 days before they are automatically
removed (http://www.cs.ait.ac.th/laboratory/email/spam.shtml#spam).
If you wish to see any of the following message, reply to this
email,including the lines with the File: information below.
The word File: MUST be in your reply message, along with the filename.
".$body;
open MESS, "|/usr/sbin/sendmail -t";
print MESS "From: quarantine\@cs.ait.ac.th\nTo: $user\@cs.ait.ac.th
Reply-To: quarantine\@cs.ait.ac.th\nSubject: Quarantined messages\n\n";
print MESS $body;
close MESS;
}
}]]> |
|
Perl script to prepare the summary and move the messages to the quarantine directory
It only applies to users with a groupid in the range [20-99] which
for me corresponds to real users.
It will locate every file in /var/mail that have a name in the format
user-spam.date.pid and extract the Subject:,
From: and Date: headers from the message. This along with
the filename consists of the summary of a quarantined message.
Once it has been summarized, the quarantined message is moved to /var/spamassassin/spam.
To avoid that SpamAssassin tries to proceed the summary email, I added the
following rule at the begining of the procmail configuration file:
| |
<![CDATA[
:0 :
* ^From root
* ^Subject: Quarantined messages
* ^Reply-To: quarantine@cs.ait.ac.th
$DEFAULT]]>
| |
Example of procmailrc shortcut SA
Every email sent by root, with the Subject: Quarantined
messages and a Reply-To: quarantine@cs.ait.ac.th is delivered
immediately in the users' $DEFAULT mailbox.
This script run by crontab, as often as you deam necessary. But it should not
be so often as it would generate one summary per spam message. Else for the
user it would defeat the purpose. I found out that once every 6 hours is fine
for me.
The user will receive a summary that looks like:
| |
<![CDATA[
Date: Mon, 18 Mar 2002 16:42:35 +0700 (ICT)
From: quaratine@cs.ait.ac.th
To: on@cs.ait.ac.th
Reply-To: quarantine@cs.ait.ac.th
Subject: Quarantined messages
You have received email(s) that is suspicious spam and was quarantined.
Quarantined messages are kept for 30 days before they are automatically
removed.
If you wish to see any of the following message, reply to this
email,including the lines with the File: information below.
The word File: MUST be in your reply message, along with the filename.
1 File: on-spam.200203181540.77885
Spam Level: 9.3
Date: Mon, 18 Mar 2002 15:40:00 +0700 (ICT)
From: Mail Delivery Subsystem <MAILER-DAEMON>
Subject: Postmaster notify: see transcript for details]]>
| |
Example of summary for quarantined spam
Step three, allow users to recover quarantined message
Users can recover any quarantined message, by sending an email to quarantine@cs.ait.ac.th
where he mentions the identifier of the message you want to recover. Identifier
of a quarantined message is a line of the form:
File: user-spam.date.pid.
| |
<![CDATA[
To: quarantine@cs.ait.ac.th
In-reply-to: <200203180942.g2I9gZM82106@mail.cs.ait.ac.th>
(quaratine@cs.ait.ac.th)
Subject: Re: Quarantined messages
References: <200203180942.g2I9gZM82106@mail.cs.ait.ac.th>
--text follows this line--
> 1 File: on-spam.200203181540.77885
> Spam Level: 9.3
> Date: Mon, 18 Mar 2002 15:40:00 +0700 (ICT)
> From: Mail Delivery Subsystem <MAILER-DAEMON>
> Subject: Postmaster notify: see transcript for details]]>
| |
Example of email send for recovering a quarantined message
Such email are procced by a special receipt in procmail:
| |
<![CDATA[
:0
*^TOquarantine
| /usr/local/sbin/recover-spam
:0 :
* ^From quarantine@cs.ait.ac.th
$DEFAULT
]]>
| |
Example of procmailrc call the recovery script
The first receipt calls the Perl script /usr/local/sbin/recover-spam
for every email sent to qurantine. The second receipt makes sure that
an email recovered by the following procedure is pushed without further checking
in SA.The Perl script is as follow:
| |
<![CDATA[
#!/usr/local/bin/perl
while (<STDIN>) {
# skip the header, until a blank line, keep only the From:
$from=$1 if /^From\s+([\w\.\-]+)/;
last if /^$/;
}
while (<STDIN>) {
next if /File: information/;
next if /File: MUST/;
next unless /File: ([\w-\.]*)/;
# in the message body, locate every line with
# "File:" followed by a valid file name of the form
# user-spam.date.pid
$file=$1;
$user=$file;
$user=~s/-spam\.\d+\.\d+//;
next unless $user eq $from;
# check that the user mentionned in the filename
# is the same as the user that sent the email
# don't allow one to recover messages from others
$file="/var/spamassassin/spam/$file";
next unless -f $file;
system "/usr/bin/procmail -f quarantine\@cs.ait.ac.th -Y -d $user<$file";
# use procmail to push the file in the users' mailbox
unlink $file; # remove the file as it has been delivered]]>
| |
Perl script to recover quarantined message
The script skips the headers, it only keeps the username of the sender of the
recover message, for control purpose later on.
Then in the body of the message it will locate every line of the form
File: user-spam.date.pid.
and will push the corresponding file in the users' mailbox, only if the user
is the same as the sender of the recover message.
Once a quarantined message has been recovered, the file is removed.
Step four, expire quarantined messages
Old quarantined messages are expired when they are more than 30 days old. The
following Perl script will do the job:
| |
<![CDATA[
#!/usr/local/bin/perl
$interval=31*24*60*60; # 2,678,400 seconds
$today=time();
chdir "/var/spamassassin/spam";
opendir DIR, ".";
while ($file=readdir DIR) {
next unless -f $file;
next unless $file=~/-spam\.\d+\.\d+$/;
($a, $b, $c, $d, $e, $f, $g, $h, $i, $mtime, $r)=stat $file;
next unless $today-$mtime>$interval;
unlink $file; # remove any file older that $interval seconds
}
]]> | |
A perl script to expire old quarantined spam
This is run by cron, as often as you deam necessary.
Complete synopsis of procmail configuration file
To sumamrize, the procmail configuration file is as follow:
| |
<![CDATA[
# recover quarantine messages
:0
*^TOquarantine
| /usr/local/sbin/recover-spam
# email pushed by the quarantine recovcery are delivered immediately
:0 :
* ^From quarantine@cs.ait.ac.th
$DEFAULT
# do not check the summary messages
:0 :
* ^From root
* ^Subject: Quarantined messages
* ^Reply-To: quarantine@cs.ait.ac.th
$DEFAULT
# check one message with smapc/spamd
:0fwE
| /usr/local/bin/spamc -u $LOGNAME
:0e
{
EXITCODE=$?
}
DATE=`date +%Y%m%d%H%M`
PID=`echo $$`
SPAMFILE=$DEFAULT-spam.$DATE.$PID
# quarantine in /var/mail/user-spam.date.pid
:0:
* ^X-Spam-Status: Yes
$SPAMFILE
]]> | |
