What the !#$@ do I do with the headers?

As a follow-up to my previous post published recently on Deliverability.com presenting how to find email headers in the most common email clients available, here's a high level overview of what exactly those headers mean. 

You've found where the headers are.  Great!  But, if you're not sure what they mean or where to go next, hopefully this post will clear things up a bit.  As an email professional, you should always check headers of actual email messages sent out as a test to your own account before launching the email into production and also by seeding random live messages.  It's very easy to overlook key meta data describing your email – it's on every message that goes out, by nature of the underlying technology around email transmission (SMTP).  Not only can they give you evidence of how your email is seen by receivers but they are also great for diagnosing how ISP's treat your mail.

Since nothing is standard except the most low level layers around email communication, you might have to hunt around depending on which client is used for receiving the message and which headers you're looking for.  Unfortunately, nothing fits into a "one size fits all" header mapping.  But, there is hope.  If you understand what to look for and get used to seeing the different types of headers that are displayed (you must practice looking at different headers), you'll quickly be able to identify the key areas where your message may be running into problems.

Let's break it down.  The following is a random message I picked out of my personal Gmail account from Plaxo urging me to reengage with the site.  I picked Gmail because it does a good job of showing all headers in native form without much munging.  Headers will look different depending on what service you're using to collect mail!  But the underlying intent of the header should remain the same.

1.  Delivered-To: The address of the mailbox the email was actually delivered to when rendered.  This is usually the same as the To: recipient but in some cases can be different if you have forwarding rules enabled.

2.  Received:  A very valuable data point which explicitly tells you the path the message took from sender to receiver by denoting the machine hops it made in between.  It is made up of IP addresses and/or server names, which in conjunction, spell out the sending identity and the receiving machine and domain.  If you want to get really crafty, you can begin looking up the hops along the way to tell what networks the email touched during its journey.  This is also very helpful when trying to determine exactly what the receiving machine can tell about where the email originated.  And, you might have guessed, this is where the IP reputation is garnered and thus assigned back to a sender. An old but good post from AOL about IP reputation here.

3.  Return-Path:  The return address for the email if it bounces.  Note, this is different than the Reply-To: in that this is the required address the receiving server can communicate back to the sender for whatever reason whereas Reply-To: denotes where a message will be sent when a recipient hits "Reply".  These two can often be the same, though.

4.  Received-SPF:  The determination listed in this header will tell you if the message passed SPF authentication.  If not, then you need to revisit the From: address listed on the message and ensure it resolves to a domain that the IP address is allowed to send from (the IP address found in the Received: header).

5.  DomainKey-Signature:  Is your signature correct?  If so, the end result is your email passes DomainKey authentication.  If not, then you need to revisit the authentication parameters you're using.  It's better to send a message not authenticated than one that fails authentication.  The signature is the value your sending server calculated based on the content and headers (encryption based authentication).

6.  DKIM-Signature:  Same as with 5, this is where the newer domain authentication protocol is shown to pass or not.  DKIM is backwards compatible with DomainKeys.

7.  Content-Type:  You've probably been told you should make sure the email content you send is represented properly by the headers of the message.  This header tells the receiving system what sort of content to expect in the body.  Text/plain is used when no HTML is present but multipart is best as it has both a text and HTML part which allows for greater render success depending on what the recipient is using to view the mail.

8.  X-<value>:  An X-header is simply a header the sending or receiving server can insert without it interrupting the mail flow.  Think of it as a comment, wherein it can give you useful information about antivirus software used, versions of the email platform, ESPs used in some cases, or spam scan results but the inclusion or absence of the header itself won't impede the delivery of the message.  Depending on how verbose the systems talking are, this section can grow to be very long.

9.  Message-ID:  This is the email fingerprint and can be very helpful when communicating problems with email receivers about a specific, tractable problem where an actual email example is needed.  Remember, most receivers get billions of messages a day depending on their user base so having a pointer back to actual evidence comes in very handy when mitigating deliverability problems.

10.  Standard headers:  These are the basic headers we should all be used to seeing and what's usually shown in all email clients.  The friendly from, from address, subject, date/time and who the message is addressed to can be found here.  It is also important to ensure your headers are not intentionally misleading to comply with CAN-SPAM.

There is a distinction between what is called an "envelope" and a "message" header but I will forgo that here since I doubt many folks reading this blog care too much about the semantics.  However, if you want to know about headers, go here to see a full listing in detail of the machine language headers that are inserted into email.

There you have it.  As always, please post questions or comments if you'd like know more or there&#
39;s something I can help you understand.


3 Responses to “What the !#$@ do I do with the headers?”

  1. J.D. Falk
    April 19, 2010 at 3:23 pm #

    Good start, but some of this may be a bit misleading to novices.

    Delivered-To: and Return-Path: are fairly common, but not all MTAs insert them.

    Spammers have been inserting fake Received: headers for years, so reputation is more commonly based on the actual SMTP connection characteristics rather than parsing headers after the fact.

    Authentication-Results: is more relevant to troubleshooting than Received-SPF (which is rare) or the DomainKeys or DKIM signature headers, because Authentication-Results shows the actual results of the authentication checks.

    And, don't forget that fake headers can always be inserted by bad guys. Some systems detect & remove some fakes, but most leave 'em alone — because for delivery, it's the envelope that actually matters.

  2. Jeff Macdonald
    April 19, 2010 at 7:49 pm #

    Nice article Chris. One nit:

    "4. Received-SPF: The determination listed in this header will tell you if the message passed SPF authentication. If not, then you need to revisit the From: address listed on the message and ensure it resolves to a domain that the IP address is allowed to send from (the IP address found in the Received: header)."

    I'm pretty sure that the domain Google uses for this check is from RFC5321 Mail From. Most of the time that is the same as RFC5322 From, but in cases where it isn't, your advice would lead someone astray. In other words, Google is doing a normal SPF check vs a SenderID check, which uses the PRA algorithm to select the domain.

  3. Chris Wheeler
    April 20, 2010 at 1:26 am #

    Thanks, J.D. and Jeff. Great points. I tried to not get too much into the technical minutia to avoid confusing some of the less technically savvy readers.

    Headers should not be maliciously used to circumvent receiver checks and balances. And, most of them are smart enough to know when a header is being forged so you'll most likely get bad deliverability anyway. But, if you are sending legitimate email (meaning it has perceived value by the intended recipient), you should be transparent and let the sending server do its job in accurately inserting appropriate headers.

    Also, wrt to Jeff's point, SenderID is indeed different than SPF. SPF checks the HELO value against the envelope From: (usually Return-Path:) while SenderID checks against the PRA (purported responsible address). For further reading regarding the nuances between the two types, see: http://www.openspf.org/SPF_vs_Sender_ID