Recently I converted my mail server to the Synology mailserver. I use a setup with IMAP accounts for storing all my emails.
Getting a running mail server converted to Synology went quite well and quickly. I must say I am impressed with the result.
What I do find scary is the backup facility. I really need to check if I can restore this backup easily. But how do I do this, I do not have a spare Synology to try this on. Maybe a project for a rainy day, a nice virtual image with the Synology software would be nice!
But now the subject at hand. After a few days I was again receiving spam messages, something I had overcome on the previous mailserver using assp. This should be a problem easily tackled on the Synology I thought.
Just go into the UI and check “Enable SpamAssassin filter engine” and boom, no spam.
The engine did not catch much… Okay maybe it needs further setup. Just go into Settings and enable auto learning?
Well as I have found before, Synology is a looker but often not a performer. I need to dig into the internals.
I was amazed to find on the web that spamassassin is bundled with some amazing tools. One that solves another of my long wished for features. How to you manage spam as a mail user without needing to access a mail server.
The solution is simple, each user on the mail server need to have a standardized Junk email folder in their IMAP store. On the mail server we regularly schedule to update a bayesian index on every users Junk folder. Now when new spam mail comes in and is not detected a user only needs to mark it as spam on his iphone, the mail is moved to Junk and the server indexes this every 2 hours or so to prevent this spam from coming in again.
Theory is great but now the setup!
The tool is called sa-learn and is called in the following way:
/var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/
The last parameter is the magic trick to look at all the Junk mailboxes. Great, only it does not work on Synology….. 🙁
This is due to perl configuration. But fear not, others have found a solution, you need to call it as follows:
perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/
Problem is this will probably break on the next major DSM update but we will cross that bridge when we come to it.
The tool has some more tricks, so what I do is create the a script with name /opt/spamscan.sh with the following contents:
#!/bin/sh
date
echo "Scanning Spam Folders(s)"
perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/
echo "Updating SpamAssassin DB"
perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --sync
echo "Backing up SpamAssassin DB"
perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn --siteconfigpath /var/packages/MailServer/target/etc/spamassassin --dbpath /var/spool/MailScanner/spamassassin/ --backup > /opt/spamassassin_backup.bak
date
Use chmod on the script file to allow executing. chmod u+x /opt/spamscan.sh chmod g+x /opt/spamscan.sh chmod o+x /opt/spamscan.sh And I schedule this using /etc/crontab: 0 */2 * * * root /opt/spamscan.sh >> /var/log/spam_learn.log Wow, now all ready and done. You need to be patient. Having 20 Junk mail messages is not good enough for bayesian statistics. About a month later I had a few hundred but was not seeing all my spam being detected. Hmm BIG disappointment! Lets dig further. If you use your mail client to look at messages you can see there are added headers in the format of: X-Mailscanner. I found that some messages have a line X-Mailscanner-Spamcheck: not spam, SpamAssassin (not cached, score=3.6, required 5, BAYES_99 3.50, RDNS_NONE 0.10) but others do not. This is not working as I expected. I also found that the Mailscanner has 5 running instances on my Synology all taking about 50 Mb of memory, I have only limited mails coming in every day so I need to tone this down, the Synology only has 512 Mb… Okay lets dig into this and find the config files for SpamAssassin and Mailscanner. You can find these in /var/packages/MailServer/target/etc/MailScanner. But be careful changing this file, on next reboot it will be overwritten bij de config template. That is wat needs to be changed. So get your self a cup of coffee and use your favorite editor to update the file /var/packages/MailServer/target/etc/template/mailscanner.template, there are hundreds of options. Also nice to know, logging for the mail server is sent to /volume1/@maillog/ and not /var/log. The items I modified are:
%org-name% = <your name here>-
Max Children = 3
Always Include SpamAssassin Report = yes
Multiple Headers = add
Place New Headers At Top Of Message = yes
Log Spam = yes
Log Non Spam = yes
And now after running a while the results are much better. When a spam messages gets through, the headers will explain why, usually because spam score is not high enough. I add this email to Junk and the next identical spam message will always have a higher score until it is recognized correctly. I am a happy camper.
Thank you Synology for providing the platform and thank you internet for helping to configure the damn thing!
Hi
I did use the script and it did work so far.
Now I got an upgrade to Diskstation V5
And the script does not work anymore.
/var/packages/MailServer/target/bin/sa-learn –siteconfigpath /var/packages/MailServer/target/etc/spamassassin –dbpath /var/spool/MailScanner/spamassassin/ –spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/
defined(%hash) is deprecated at /var/packages/MailServer/target/share/perl5/vendor_perl/Mail/SpamAssassin/Dns.pm line 762.
(Maybe you should just omit the defined()?)
Learned tokens from 0 message(s) (0 message(s) examined)
server> perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/5.8.6 /var/packages/MailServer/target/bin/sa-learn –siteconfigpath /var/packages/MailServer/target/etc/spamassassin –dbpath /var/spool/MailScanner/spamassassin/ –spam /vo
lume1/homes/*/.Maildir/.Junk/{new,cur}/
defined(%hash) is deprecated at /var/packages/MailServer/target/share/perl5/vendor_perl/Mail/SpamAssassin/Dns.pm line 762.
(Maybe you should just omit the defined()?)
Learned tokens from 0 message(s) (0 message(s) examined)
Any Idea? There are lots of spam in the spam folder but nothing learned.
I haven’t upgraded to diskstation v5 yet, reading your post I am glad to wait! Maybe somenone else can help?
My cron job seems to be working fine. However when I use the following command:
/var/packages/MailServer/target/lib/perl5/core_perl /var/packages/MailServer/target/bin/sa-learn –dumb magic
The scan shows that thousands of messages are being scanned but nothing seems to be learned. I am wondering if this is because I am running the sa-learn as root on my cron whereas the “mailscanner” process is run as “postfix” in the following config file:
/var/packages/MailServer/target/etc/template/mailscanner.template
0.000 0 3 0 non-token data: bayes db version
0.000 0 0 0 non-token data: nspam
0.000 0 0 0 non-token data: nham
0.000 0 0 0 non-token data: ntokens
0.000 0 0 0 non-token data: oldest atime
0.000 0 0 0 non-token data: newest atime
0.000 0 0 0 non-token data: last journal sync atime
0.000 0 0 0 non-token data: last expiry atime
0.000 0 0 0 non-token data: last expire atime delta
0.000 0 0 0 non-token data: last expire reduction count
To get this all to work I had to create the following symbolic in the “/volume1/homes/root/.spamassassin” directory to the files in the “/volume1/@MailScanner/spamassassin directory”
once the links were created my the –dump magic started showing the correct progress.
I also deleted the old auto_whitelist file from the @MailScanner folder to allow the system to start fresh. Too many spammers made it into the auto_whitelist file with the previous setup.
Here is my cron:
TODAY=$(date)
echo “———— $TODAY ————”
echo “Scanning Spam Folders(s)”
/usr/bin/perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/core_perl /var/packages/MailServer/target/bin/sa-learn –no-sync –siteconfigpath /var/packages/MailServer/target/etc/spamassassin/ –dbpath /var/spool/MailScanner/spamassassin/ –progress –spam /volume1/homes/*/.Maildir/.Junk/{new,cur}/
echo “Scanning Ham Folders(s)”
/usr/bin/perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/vendor_perl,/var/packages/MailServer/target/lib/perl5/core_perl /var/packages/MailServer/target/bin/sa-learn –no-sync –siteconfigpath /var/packages/MailServer/target/etc/spamassassin/ –dbpath /var/spool/MailScanner/spamassassin/ –progress –ham /volume1/homes/*/.Maildir/{new,cur}/
echo “Syncing”
/usr/bin/perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/vendor_perl,/var/packages/MailServer/target/lib/perl5/core_perl /var/packages/MailServer/target/bin/sa-learn –sync
echo “Current Status”
/usr/bin/perl -T -Mlib=/var/packages/MailServer/target/lib/perl5/vendor_perl,/var/packages/MailServer/target/lib/perl5/core_perl /var/packages/MailServer/target/bin/sa-learn –dump magic
echo “”
Hi, can you explain this?
To get this all to work I had to create the following symbolic in the “/volume1/homes/root/.spamassassin” directory to the files in the “/volume1/@MailScanner/spamassassin directory”
Even if i create the root folder in homes, which is missing on my setup, I can’t get sa-learn –dump magic to work.
Thanks
figured it out 🙂
it should be:
mount -o bind /root/.spamassassin /volume1/@MailScanner/spamassassin
not:
mount -o bind /volume1/homes/root/.spamassassin /volume1/@MailScanner/spamassassin
I am using a DS213+ DSM 5.0-4493 Update 2
I created symbolic links for the following two files in the “/volume1/homes/root/.spamassassin/” directory
lrwxrwxrwx 1 root root 45 Jul 4 23:09 bayes_seen -> /volume1/@MailScanner/spamassassin/bayes_seen
lrwxrwxrwx 1 root root 45 Jul 4 23:09 bayes_toks -> /volume1/@MailScanner/spamassassin/bayes_toks
Here is what I am getting from my -dump magic per the /var/log/spam_learn.log
———— Mon Jul 21 02:00:01 CDT 2014 ————
Scanning Spam Folders(s)
Learned tokens from 3 message(s) (1479 message(s) examined)
Scanning Ham Folders(s)
Learned tokens from 9 message(s) (11610 message(s) examined)
Syncing
Current Status
0.000 0 3 0 non-token data: bayes db version
0.000 0 1667 0 non-token data: nspam
0.000 0 1072 0 non-token data: nham
0.000 0 167670 0 non-token data: ntokens
0.000 0 1404800145 0 non-token data: oldest atime
0.000 0 1405925968 0 non-token data: newest atime
0.000 0 1405927114 0 non-token data: last journal sync atime
0.000 0 1405927137 0 non-token data: last expiry atime
0.000 0 691200 0 non-token data: last expire atime delta
0.000 0 27603 0 non-token data: last expire reduction count
although I have my sa-learn working my system does NOT seem to be using BAYES. Here is a copy from one of my emails:
X-MailScanner-SpamCheck: not spam, SpamAssassin (not cached, score=0.642,
required 5, HTML_FONT_LOW_CONTRAST 0.54, HTML_MESSAGE 0.00,
RDNS_NONE 0.10, SPF_HELO_PASS -0.00, SPF_PASS -0.00)
Any idea how to enable BAYES?