Spam Filtering Concepts 201

Spammers are getting more and more aggressive every day. If you have a publicly posted email address, I'm sure you've gotten a lot of email spam including phrases like the following.

VIAzGRA
CIAzLIS
AMBzIEN

The phrases are regularly re-randomized. A million junk email messages might go out with the combinations shown above, but two weeks later the random character will become a 'Q' and then shift two letters to the right.

The SpamButcher spam blocker application works on fuzzy logic principles. A simplified explanation of this is that it uses a fancy points-based system. "Buy Now" might be worth 200 points. "Click Here" might be worth 250. The scores are totaled, a little extra magic happens, and then the program decides if the message should be filtered.

With examples like the above this doesn't work so well. The letters "VIA" and "GRA" might appear in each message, but they are broken up with random letters, and sometimes even randomized HTML.

Making things tougher, those letter combinations do occur elsewhere in the language, and sometimes just at random inside attachments, or due to typos.

The solution still falls back to augmenting the existing approach to prevent spam email. Messages which contain lines starting with "VIA" and "CIA" and "AMB" are statistically extremely unlikely, and are penalized with a large number of points. Those individual terms may garner some points, but not enough to filter a message in general.

Back