How Email works…part Deux

So, I know I never did get to the second post of this series as soon as I wanted, but I also hadn't expected so much travel to pop up between the time of the first post and now.

As you hopefully recall from 30 days ago… we wanted to break into sections how email works

• How MTAs (Mail/Message Transfer Agents) Communicate
• What is a “Bounce?”
• How MTAs Handle Bounces
• How Bounces Can Affect a Subscriber’s Status

Today we are taking on the second part of this series What is a "Bounce"

So what is a bounce? One aspect of good Email Marketing practice is being able to interpret and respond to the machine responses (referred to as “bounces”) when email fails to be delivered to its intended recipient. Bounces occur because the recipient's mailbox is full, the mail server is temporarily unavailable, or the recipient no longer has an e-mail account at that address. Bounce usually appears as a new note in your inbox when you send a personal or work related email.
 
Typically, a bounce message will contain several pieces of information to help the original sender in understanding the reason their message was not delivered:

• The date and time the message was bounced
• The identity of the mail server that bounced it
• The reason that it was bounced (e.g. user unknown or mailbox full)
• The headers of the bounced message
• Some or all of the content of the bounced message

AOL has a good example of one here.

For most ESP's and large bulk senders, the bounces that are sent back include a codified message in the form of a  numeric series (550 or 421) and an expanded description sometimes called a DSN (Delivery Status Notification, such as “User Unknown” or “You’re blocked”). In order to comply with the established ISP Acceptable Use Policy (AUP) and to further distinguish your emails from spam, many sending systems like Eloqua and MTA providers like StrongMail have established efficient and updatable capture and reporting mechanisms for reading and accurately interpreting these bounce messages and then report on them in a categorical way that might normalize the data and organize it into logical categories, such as hard (permanent) bounce, soft (transient) bounce, block, and technical failure. As my industry colleague and friend Dave Lewis said in an article, "a good system will then map these delivery failures into subcategories, such as “unknown user” under hard bounce or "URL block" under block". By interpreting or categorizing these bounces, actions can be taken in regard to the bounces and will range from unsubscription of invalid addresses on the first hard bounce to limiting mailing to a domain where a block or some other form of service denial has been detected to removing bad or unknown user addresses.

So what does the number mean? what does the words means?

Well the numbers basically are a way to tell a mail server to re-try or NOT re-try an email if it fails.

Anything that is 4xx like 421 is a temporary failure and can be re-tried immediately. Could be for a quick server outage, greylisting, etc.

Anything that is 5xx like 550 is a permanent failure and SHOULD NOT be re-tried immediately. Usually for a long term block or unknown user or bad email account and should be looked at later for reasoning and whether or not the email should be re-tried again based on the DSN.

The DSN or wording is just a bounce to help humans read the bounce when it comes back to your email client vs. an ESP bounce system. Tells you more as to why it bounced.

So some examples:

• 550 User Unknown = Don't re-try now and don't send again
• 421 Greylisting = Re-try now and send again
• 550 Your blocked = Don't re-try now and send later after addressing why block occurred
• 451 no reverse DNS = Re-try now and send again later after you fix your DNS
• 552 message to big = Don't re-try now, but send again later after reducing size

and the list of examples go on.

One thing to consider here; capture all data streams. Make sure that your system collects all bounce data streams – both synchronous (sync) and asynchronous (async) bounces. Synchronous bounces happen immediately during the email transmission. For example, if your sending system connects to an ISP and tries to send an email to nouser@domain.com and if the ISP's mail server is configured for synchronous bouncing, it will immediately pause the transmission, and tell you that the address is bad and not allow for anymore data to be sent.

Asynchronous bounces work in the opposite way. Using the example above, domain.com would accept your email and try to deliver it to its internal address records. Once it realizes that the email address is bad, the ISP's mail server will then send your mail server an email letting you know. You asking yourself why do it this way?  Spammers often use a form of dictionary attack, sometimes known as a Directory Harvest Attack, for e-mail address harvesting. For example, a spammer may try sending messages to adam@example.com, barbara@example.com, carl@example.com, etc. Any addresses to which messages are delivered, as opposed to being bounced back during the first half of the send, can be added to the spammer's list of known-valid addresses if it doesn't bounce in the sync process. So many ISP's move that bounce processor till later as an async so they can monitor those who have spammer looking activities. Statistically we see more ISP's bouncing email synchronously.

Now another thing that you need to consider here is that when the standards for how bounces should be sent and used was created to ONLY address technical failures such as server outages, full mailbox's, bad email address, etc. It was never intended to handle policy based bounce such as spam or phishing blocking. This wasn't something intentional, we just never though this far ahead nor could we. In MAAWG we are looking at this now to understand how we can build to fix issue like this on the existing infrastructure. Come to San Francisco MAWAG to see!

TIPS:

Make sure when creating or buying into an ESP or in-house solution that the bounce management system can

1. Capture all data streams such as syns and async
2. Correctly interrupt the data that can be confusing in itself in the above bullet examples
3. Organize and normalize the data to handle the upon 1,000's, upon 1,000 of bounces you might get
4. Make the data and reporting actionable to the user. Remove the email address? re-try the email?
5. Be updateable since bounces can change over time and ISP's are always changing.

Again, Dave Lewis said "With a bounce management system that meets these requirements, you’ll be in a position to properly evaluate your performance, manage your list and improve your practices – all of which translate into better bottom-line results. And you’ll be well on your way toward mastering e-mail deliverability"

MORE information can be found here http://en.wikipedia.org/wiki/Bounce_message

-Dennis
Eloqua

Don't Just Send, Deliver!

Last 5 posts by Dennis Dayman

• Delivery and deliverability debunked - May 7th, 2012
• European Regulator Warns Silicon Valley About Privacy - April 28th, 2012
• Canada’s anti-spam law won’t take effect until 2013 - April 28th, 2012
• Mr. Dayman Goes to Washington - April 2nd, 2012
• EU proposes a reform of the data protection rules - March 28th, 2012