Keeping Critical Systems Up 99.9% of the Time - Email

(this article is part of a series)

Designing your email system to be fault tolerant only requires that you follow a few basic guidelines. The SMTP protocol was designed to provide redundancy - all you have to do is use it.

It should be noted that this document may help you get close to 99.9% uptime for your email system - but that's not the same as reliably receiving 99.9% of email. To do so has to take into account things like false positives by anti-virus and spam email programs. Due to massive amounts of spam, scams and viruses email has been seriously compromised over the last decade. The whole system may be in need of an overhaul to restore any notion of true reliability.

Each domain should be configured with at least two incoming SMTP servers on different networks
As so frequently happens, ISPs and system administrators try to save a little money. They may use only a single email server. Often they'll setup two different DNS records pointing to the same server, or host the backup server on the same network as the primary server. These configurations don't really provide good redundancy.

Incoming SMTP servers are configured in a domain's DNS as "MX" (mail exchange) records.

Each MX record has a mostly-arbitrary preference or priority number associated with it. The lower the number, the "higher" the priority.

For instance - SpamButcher's MX records look like:

MX preference = 5, mail exchanger = linode.spambutcher.com
MX preference = 20, mail exchanger = mail1.no-ip.com
MX preference = 25, mail exchanger = mail2.no-ip.com

Wouldn't it be nice if they'd implemented something similar when specifying the standards for other services? Imagine if setting up an automatic backup web server only required specifying a secondary DNS entry.

Find a backup SMTP server, and make it do something reasonable
Your ISP might offer a good backup mail configuration, but there's a good chance they don't.

Again, I'll recommend looking at the services offered by No-IP. For about $30 / year they'll provide backup MX services for your domain. If your main server goes down, their servers will spool any incoming mail for up to a week. When you get your main server back online, their backup servers will send any queued email to it.

As an alternative, your existing DNS provider may offer basic mail-forwarding. By setting up their server as a lower-priority MX record, you might be able to use their services as a backup.

You can setup any critical accounts to forward to logical POP3 accounts hosted someplace other than your main email server (like a GMail or Yahoo! account). This lets you continue to receive and read email even while your main SMTP and POP3 servers are down. However, this configuration can become complicated if dealing with more than a few users.

Minor "hiccups" in connectivity do happen. You should assume that periodically a little mail might go to your backup server, even if you don't have any major outages. This is to say, you don't want your backup server forwarding your messages to a POP3 account you only check once a month.

Verifying that your backup SMTP server configuration actually works is a good idea, but can be a bit tricky. The main point is to send a test message directly to your backup SMTP server, and verify that the email goes someplace logical.

You may be able to reconfigure your email client to target a message at your backup SMTP server for testing. If that fails - you may want to study up on SMTP commands, and try sending a test message manually via telnet.

Next article: Ecommerce

Back