Home Page
Archive > Posts > Tags > Email

Useful Exim Scripts
For fighting spam

In the course of my Linux administrative duties (on a cPanel server), I have created multiple scripts to help us out with Exim, our mail transfer agent. These are mostly used to help us fight spam, and determine who is spamming when it occurs.

This monitors the number of emails in the queue, and sends ours admins an email when a limit (1000) is reached. It would need to be run on a schedule (via cron).
export Num=`/usr/sbin/exim -bpc`
if [ $Num -gt 1000 ]; then
        echo "Too many emails! $Num" | /usr/sbin/sendmail -v "$AdminEmailList"
        #Here might be a good place to delete emails with “undeliverable” strings within them
        #Example (See the 3rd script): exim-delete-messages-with 'A message that you sent could not be delivered'

This deletes any emails in the queue from or to a specified email address (first parameter). If the address is the recipient, the from must be "<>" (root)
exiqgrep -ir $1 -f '<>' | xargs exim -Mrm
exiqgrep -if $1 | xargs exim -Mrm

This deletes any emails in the queue which contain a given string (first parameter)
if [ "$1" == "" ]
  echo 'Cannot delete with empty string'
  grep -lir "$1" /var/spool/exim/input/ | sed -e 's/^.*\/\([a-zA-Z0-9-]*\)-[DH]$/\1/g' | xargs /usr/sbin/exim -Mrm

Get a count of emails in the queue per sender (sender email address is supplied by sender and can be faked)
exim -bp | grep -oP '<.*?>' | sort | uniq -c | sort -n

Get a count of emails in the queue per account (running this script can take a little while)
exim -bp | grep -Po '(?<= )[-\w]+(?= <)' | xargs -n1 exim -Mvh | grep -ioP '(?<=auth_sender ).*$' | sort | uniq -c

Bonus: Force all non-specified accounts on Exim to use a certain IP address for sending. It would need to be run on a schedule (via cron).
export IPAddress="YOUR ADDRESS HERE"
/usr/bin/perl -i -pe 's/\*:.*/*: '$IPAddress'/g' /etc/mailips
Always make sure your envelope sender is correct
Otherwise things may not work as you expect

When you send an email there may be multiple fields in the email header that specify the email address that it came from and how to reply back to that address. Some of these are:

  • From: This is the field that the user sees in their email client as the "From" address. This field is the most easily (and most often) spoofable as you can put anything you want in this field and it doesn't change how the email is received or responded to. Most systems, in my experience, don't try to protect this field either.
  • Envelope sender: This is used internally by email software to see who the email was really from. Different systems (i.e. spam blockers) can use this field for different purposes.
  • Return path: This field specifies the email address to reply to when you click the "reply" button on your email client.

There can be multiple problems if the latter 2 field are not properly set. Some of these are:
  • Spam blockers may be more likely to identify the email as spam
  • The email might be sent from the wrong IP address. Exim (which cPanel uses by default) might be configured to check /etc/mailips to determine what IP address to send from depending on the domain of the envelope sender.
  • The recipient might reply to the wrong email address when replying to the email.

When sending an email from PHP via the mail function through Exim you can only manually set the "From" header field (of the three) through the "additional_headers" (4th) parameter. This might be possible to remedy on some systems however.

If your server is configured to allow it (it may require privileged user permission), you can pass to the "additional_parameters" (5th) parameter of the mail function the -f Exim option, which sets the envelope sender and return path. For example:

mail('example@gmail.com', 'This is an example', 'Example!', 'From: example@yourdomain.com', '-f example@yourdomain.com');

On a related security note, if you think an email may not be legitimate, don't forget to check the email headers by viewing the original email source. Our servers include many useful headers in emails to help combat fraud including (depending on circumstances) the account the email was sent from, the IP address it was sent from, if it was sent from PHP, and if so, the script it was sent from.

PGP for Thunderbird in Windows
Securing your communications is important

I have been using and recommending Thawte’s free SMIME email certificates for a number of years. Personal email certificates have always been a bit more attractive for me than PGP for communicating securely with other [not always very computer literate] people since they are directly integrated into most [if not all] email clients.

It’s always been nice to be able to sign an email to another person with my certificate, and just tell them to hit the “encrypt” button next time they send me something either of us want encrypted :-) (email clients automatically store a certificate after receiving it, which signing includes).

Thawte has been my choice for SMIME certificates because they issue them for free, and it’s not something that’s really worth paying for. There are other services out there that do the same kind of thing for free, but Thawte is a large name I trust. Unfortunately, Thawte recently announced that they are closing down their free email certificate program.

I have been using PGP for a few years to communicate with multiple people too, and have decided to try and move all my friends/clients over to it too due to the circumstances. Also, PGP has the major advantage of you creating your own private keys instead of a 3rd party (i.e. Thawte) doing it, meaning only you have the key to access emails encrypted to you.

So anywho, here’s the info on getting PGP set up with Thunderbird in Windows for anyone that needs it.

  • First, of course, you’ll need Thunderbird, which can be downloaded here.
  • I recommend you always send all your emails in both HTML and Plain Text, so you can have rich text formatting in your emails by default, but lame people that don’t have clients that read HTML are ok too. To do this, go to Menu > Tools > Options > Composition > General > Send Options > In the top box change it to “Send the message in both plain text and HTML”.
  • Next, you need to install PGP. I recommend using GnuPG (windows version).
  • When you run GnuPG for the first time, it’ll ask you if you want to generate a key, which you’ll want to do, unless you already have one made that you need to import.
  • Next, you’ll want to install Enigmail for Thunderbird. After downloaded it, in Thunderbird, go to Menu > Tools > Add-ons > Extensions > Install, and open the .xpi file.
  • After Thunderbird restarts, go to Menu > OpenPGP > Setup Wizard and step through it. During this setup, I personally suggest changing the following default options:
    • “Do you want to change a few default settings...” > Yes > Details > Uncheck the following
      • Disable flowed text
      • View message body as plain text
      • Use 8-bit encoding for message sending
      • Do not compose HTML message
  • To encrypt mail to other people, or verify a signed message from them, you need their public key file. Some of the ways they can send their public key to you are as follows:
    • A normal file send, in which case you will need to import it through GnuPG.
    • You might also be able to retrieve it from a public key server if they put it there, but I am not going to go into that.
    • If they send it to you through an attachment in an email, and you double click on it in Thunderbird, you will receive a prompt asking if you’d like to import the key.
  • To encrypt an email to another person, after having their public key, simple go to Menu > OpenPGP > Encrypt Message in the compose window. Make sure to also check Menu > OpenPGP > Use PGP/MIME for This Message so it can send the HTML!
  • To send your public key to someone go to Menu > OpenPGP > Attach My Public Key in the compose window.
Info on a spam filtering solution

I was long ago pointed to SpamSoap by a friend who helped lead the IT department of a rather large and prestigious law firm. It seems to be an excellent spam filtering solution, but can get to be rather expensive as it is a pay per month per mailbox program, kind of (you pay in groups, like 1-5, 6-10, ..., 201-250, etc). I wouldn’t mind too much trying out the filtering with Google’s domain email program, but Google has marked multiple legitimate emails as spam in my Gmail account in the past, and I don’t trust their cloud computing approach too much with my data.

I originally set up a SpamSoap account 2 to 3 years ago for a single client, and have more recently been setting it up for myself, family, and some other clients. The client that has been using it for that time has been very happy with it, and the only reason I didn’t start using it for myself and others at that time was because it marked a legitimate email as spam for me, and diagnosing why didn’t get very far with their tech support. I have however done a lot more research into their system recently, asking their staff lots and lots of questions to understand the system more, and believe I know why it was caught as spam. Unfortunately, their documentation is horrible and their site doesn’t really go into details at all, so information on how it all works and how to set some things up is not easy to come by. The problem, in my case, is that where the email arrives from to the SpamSoap servers is considered. Servers that SpamSoap receives a lot of spam from are marked as more likely to be sending spam, so unfortunately, forwarding emails from another address to an address on a domain filtered by SpamSoap is a bad idea, as then the whole server that manages the domain that is forwarding is marked as sending a spam message. However, this is only one of many spam determining metrics used, and, of course, it takes many spam messages to make a difference for a server, but if you are forwarding from an address that receives a lot, bad things happen :-). Anyways, here’s some of the information I gathered on how their system works, and other important tidbits, if anyone is interested in using them.

The way SpamSoap works is you pay for “user accounts”. Each user account has 1 white/black list associated with it (which isn’t technically needed, but helps things along), 1 quarantine area, and receives (if they choose too) daily quarantine reports to their master address. Each user account can have email aliases tied to it, but due to the quarantine area it’s important to separate users. Pricing is based upon number of user accounts in tiers, like 1-5=$15/month, 6-10=$25/month, 11-20=$42/month, and so on.

The actual filtering is done by setting MX records for the domains to be filtered to SpamSoap, and SpamSoap actually just sets up a proxy connection between the sending server and your server for delivery. If a message is detected as spam during this process, the delivery attempt is canceled.

There are 2 types of incoming spam that can be filtered different ways by the system; high scoring (100% likelihood) spam, and medium scoring (>90% or something like that, an exact number is not obtainable) spam. With either of these you can choose to either: Tag the message with “[SPAM]” in the header, quarantine the message, deny delivery, or let it through. There are also filtering rules and actions you can set up on based other criteria, like: viruses, content (profanity, racial insensitivity, sexual overtones, etc), click protection, and attachments.

Domain grouping with aliases is a slightly more complicated topic. You can have as many domains as you want, and it does not affect pricing; only the number of user accounts does (or if you choose other options, listed here).

Basically, first, you have master domains. A master domain can have multiple alias domains tied to it. All email addresses with the first section as the same are aliases of each other in this setup. For example, if domain1.com is a master, and domain2.com is an alias, then me@domain1.com and me@domain2.com are email aliases of each other no matter what. If you wanted to alias “myself” with that same user, then those 2 plus me@domain1.com and me@domain2.com would all be the same user. In this setup, if you wanted me@domain1.com and me@domain2.com as separate users, you would have to split up the domains to not be aliased (in a group). You cannot however alias emails across domains that are not aliased, so for example, if both domain1.com and domain2.com were master domains, you could not alias me@domain1.com and myself@domain2.com. These configuration issues really only tend to be problems with generic names like “info@” and “admin@”, for example, a problem would creep up if me@domain1.com wanted to alias myself@domain2.com but info@domain1.com and info@domain2.com needed to be separate user accounts. If this happened, domain1.com and domain2.com would have to both be their own master domains and myself&me would have to be separate user accounts and the white/black lists would need to be duplicated, and 2 quarantine reports would come in.

I would personally recommend for all normal user inboxes to have high likelihood spam denied, medium likelihood spam as quarantined, and anything with a virus as denied with a return notification. Also, anyone that wanted to not be filtered on a domain that runs through SpamSoap would need to be on one user account as aliases with the no filtering option set. The same goes for users who do not need a quarantine (freeloaders ^_^; ), in which one user account could be set up for basic filtering w/o quarantine and lists.

Because of the no forwarding problem as stated above, all domains would need to be pushed through SpamSoap with emails that needed filtered, and then they could be forwarded afterwards to the appropriate inbox from your own servers. So, in other words, domains that go through SpamSoap cannot be forwarded TO and filtered unless the domains that are forwarding to it are also set up with SpamSoap. The consequences of such are higher likelihood of anything being forwarded being counted as spam and that server being marked as a potential spammer.

SpamSoap also has separate reseller and partner programs for people that forward them business, but they would only be useful if one sent a lot of business their way, generating SpamSoap lots of revenue.

I hope that all made sense, it wasn’t easy to write out x.x; .