Inconsistencies with inbound traffic across ISPs

These results should NOT be a surprise to ANYONE here, but Return Path ran a study to find out if Internet Service Providers (ISPs) treat IPs differently when it comes to filtering decisions.

You should consider that these results are also a factor of many things such as which ISP's serve which demographics, user perception in personalities for those demographics, how spammers target different ISP's because of their differences in filtering, etc.

What's more notable in this study is that in some cases you could see a "7x difference in accepted mail from the same set of IPs when comparing their behavior at two ISPs". This should point out the fact that if your not following sender common best practices that prescribe such things as having rDNS on your sending IP's could heavily effect you at some of those ISP's. In other words some ISP's have different policies, but following the strictest of rules should help you with many delivery differences at most ISP's.

ISP's will probably for the most part continue to have delivery difference because of budgets, laws they must comply with, differences in how customers are raised, competition with each other, etc.

I think its nice to finally have a third party prove that there can be differences and that NOT all ISP's will treat your mail the same. Not trying to say it's right or wrong, but that your customers should be aware of this. It's still think it's their responsibility to handle their reputation. I would be willing to stick my neck out here today and say that most ESP's from the best common technology standards standpoint are all doing the same thing to get your mail delivered. Many ISP's have the same technology required to deliver to us standards. It's now up to you to ensure your reputation is good no matter which ISP your sending to.

——

John Young, Ph.D.

Director of Product Analytics

We encourage receiving networks to share data with us at Return Path so that we can in turn provide solutions and information that will help their filtering decisions. We believe that you can learn from another company's mistakes and success. And, when working in a collaborative environment, receiving networks can learn from cases where one system accepted mail that another system was blocking erroneously or vice versa.

We decided to dig into our data to find out if Internet Service Providers (ISPs) treat IPs differently. We took a random sample of 400,000 IPs that attempted to send messages to four different receiving networks in early 2009. The ISPs used from our network consisted of two webmail providers, one cable operation, and a hosted business email provider.

By looking at IPs that mailed to all four networks, it became clear to us that receiving networks make extremely different decisions about how to treat those mailers.

From the data, we found that when ISPs make decisions on what to do with inbound mail, they cannot agree on IPs from smaller volume mailers, especially when that IP has no rDNS. Also noteworthy was a 7X difference in accepted mail from the same set of IPs when comparing their behavior at two ISPs. Now, that's not as interesting until you note the variance in complaint rates across ISPs, which vary as much as 3X between them.

One would assume that the more email sent, the higher the probability of a user complaining. However, for this set of IPs and their corresponding data, that is not always the case; especially when looking at those smaller volume IPs with no rDNS. For example, one IP sent ~100 messages to ISPs A and B. The IP had ~90% delivered rate at ISP A, 4% Delivered rates at ISP B, but less than 0.2% complaint rates at either.

Another interesting pattern indicated agreement in the treatment of IPs with a lot of trap hits. However, there was a significant percent of IPs with a large amount of traps who ended up in the disagree buckets respectively. Again, it was mostly smaller volume mailers averaging less than ~200 messages a week.

It is evident that ISPs could benefit from sharing their experiences and their data in their fight against spam. By doing so, they could minimize the amount of spam impacting their systems, and the overall costs associated with filtering.

We are currently in the process of conducting a larger study to see if the data we found in our initial analysis holds true. If you are a receiving network who would like to participate, please contact us. In exchange for participating, you will receive detailed reporting on how your filtering differs from other ISPS as well as reputation data that would greatly improve your filtering and blocking decisions.

-Dennis

Don't Just Send, Deliver!