Doing the Laundry - Keeping SpamButcher at 98+% Effectiveness

SpamButcher is updated far more often than most users realize. In fact sometimes the product is updated several times a week.

The frequency of updates really reflects our internal development cycle. We are continually adjusting, adapting, evolving and maybe even perverting the filter to help it catch every last drop of spam. It's much easier to make these tweaks on an ongoing basis as opposed to periodically "cracking down" when it's become clear the application is in a state of disrepair.

Most updates aren't "announced." When we've made an actual bug fix, or feel there have been enough minor improvements - we'll increment the version. Most users will have the best experience if they take the time to update only when they see a newer version reported on www.spambutcher.com than they're currently running. Although, if users find themselves getting hammered by a specific new spam campaign - doing an update might help.

Tip to users - when reporting missed spam - always send it to spam@spambutcher.com - not support@spambutcher.com or any other SpamButcher address. Please don't add any additional content or commentary to the messages. If you know how to forward header information, that's also useful.

Not all spam that gets delivered to SpamButcher development gets treated the same. Some unwanted email that gets reported is "mechanically" evaluated in an effort to extract information for use in possible future versions.

However, every single piece of spam that is sent to one of our "spam trap" or "honey pot" accounts which isn't filtered gets individually evaluated by development. That's right, a real human sits down and examines the message with their own eyes.

This might sound like a waste of manpower at first, but it lets us keep the filter incredibly well tuned to ongoing fluctuations in spam campaigns. If we're doing our job, not a lot of messages make it through anyways.

Some examples of questions we ask about each missed spam message:

Did this message really get missed by the filter?
Due to SpamButcher's design, some messages are missed due to the user downloading the message before SpamButcher gets a chance to evaluate it. While this is a design problem, when tuning the filter we want to focus on spam that the program actually looked at, and incorrectly determined was not spam.

Is this message part of a larger spam "campaign?"
Sometimes thousands or millions of near-identical spam messages are sent as part of a single campaign.

We use the Usenet to help determine the scope of a given campaign's impact. Users have been reporting their spam to various Usenet groups for over a decade.

Surprisingly, some spam campaigns last not months, but years. When we see 4,000 different reports of messages using the same template spanning back to 2001 - we can safely figure that it will strike again in the future.

When it's evident a campaign is recurring - we make extra effort to block it; even if it means creating a new spam email stopper subroutine just to identify the specific campaign.

Does the message contain any other oddities that can be used to identify similar spam?
There seem to be several dozen common pieces of client-side software and infectuous malware that spammers use to send spam. These programs often "spoof" message headers in an effort to make the messages they send look normal. Some of these programs make bad choices, which can give them away. Other times these applications use templates with predictable patterns that can be recognized.

A few examples:

Did the message come from a network that's a "repeat offender?"
These days, most spam seems to be sent from botnets that can span the entire globe. I suspect within my lifetime, a computer based on a space station or moon-base will become a zombie and send spam from outer space.

Some spam is still sent from relatively traditional spammers who have servers located in centralized network operation centers with connectivity provided by a spam-friendly ISP.

When we can reliably identify these networks - they are included inside SpamButcher for future blocking.

Back