Suppose half of all doctors measure heart rate differently, and police departments calculate auto speed in multiple ways. Then, suppose the doctors and police use equipment that doesn't measure accurately.
I'm betting most of you would hope that these professions would move to ensure consistency and accuracy.
So, when Josh Baer, founder of this blog, recently questioned on Twitter the value and point of the EEC initiative to standardize and rename the open rate, I felt compelled to respond in more than 140 characters.
Here is a sampling of what Josh said:
"@LorenMcDonald while I agree with the logical argument, I think the whole thing is a waste of time. Not solving a problem. Open rate is fine."
"@LorenMcDonald when I say 'open rate' to ANYONE they know what I mean from the name. No one will know what 'Render Rate' means."
"@LorenMcDonald @Mostew who cares what the actual open rate is? How is reporting 1/3 less opens hurting me? I can still A/B split to compare."
Josh is not alone in his comments. But believe me, there are so many other things I'd rather talk about than the open rate, or, what I hope will become the recognized successor: the render rate. (Read feedback on the EEC blog and download the render rate document (PDF).)
Like the current irrational debate about single versus double opt-in, the core reasons that the EEC Measurement Accuracy Roundtable (which I co-chair) is recommending the render rate have gotten lost among the hyperbole and blog comments.
Why should you care? It starts with these essential points that are simple, straightforward and irrefutable:
- Inaccurate: The open rate has become extremely inaccurate because disabled images, use of preview panes and HTML-unfriendly mobile devices lead to an underreporting of the true number of opens. Fellow eec Roundtable member Morgan Stewart has done analysis across several ExactTarget clients and estimates a typical underreporting of from 5% to 35%. Meaning a measured 30% open rate is actually from 31.5% to 40.5%.
- Inconsistent: It's inconsistent, because different email service providers and marketing software providers calculate and report it differently. Among other things, some providers incorporate a click on a text message or HTML email where the tracking image has not loaded in their open rate calculations, and many others don't.
- Misused: It's being used to assess how well a message performed. The open rate, even putting aside its inaccuracies, does not tell you how well an email performed. It does tell you how well the combination of your "from" name, subject line, brand reputation and potentially preview-pane copy/images motivated recipients to open their email.
Oops! We don't even know if it measures that, because the email might load in the preview pane without the recipient viewing it.
Open Rate Measures Process, Not Output
Only a metric that measures whether the campaign met business goals, such as total revenue, revenue per email or conversions, can accurately reflect success or failure.
The open rate is a process metric, meaning it measures actions on the email. It's not an output metric, which measures business goals, but many marketers treat it like one.
The open rate can help assess process-related problems with your emails. But, it won't tell you whether your email delivered more revenue, leads, demos, or downloads, helped retain clients or collected more referrals.
I've met thousands of email marketers over the last several years, but not a single one has told me he or she would get a raise or bonus by increasing open rates.
I'll also agree that the open rate has its uses:
You want to use it to measure a branding campaign instead of a revenue-generator? That's fine. Newsletter publishers want to measure whether subscribers are reading their articles? OK.
Or, you are a combo brick-and-mortar-plus-online retailer such as Borders or REI that uses opens to help measure reach and the impact of email in driving people to a physical store? Got it.
Why Should You Care?
Josh, you and others should care about redefining and renaming the open rate because the average marketer still doesn't understand what an open rate is or actually measures.
In expert hands, the open rate is still inaccurate but not harmful. The metric is worrisome, however, when uninformed marketers communicate the success of their last campaign or newsletter to their management based on a few percentage-points' increase in their open rate.
If we are going to move the email industry forward – away from the batch-and-blast, just-send-more, "deliverability is someone else's responsibility" attitude of many – then ground zero starts with redefining the open rate.
Talking about the open rate might be a bore and waste of time, just like talking about blocked images, blacklists, authentication protocols and bounce rates are a poor use of a marketer's intellectual capital.
We should be talking about integrating email with other channels, lifetime customer value, cool uses of dynamic content, kick-ass trigger-email approaches and the like.
So, why are we talking about redefining the open rate? Because, until we can move the industry beyond talking about open rates as a key benchmark, we likely won't be able to talk about those higher-level matters.
Putting aside philosophical arguments about the role and value of the open rate, fundamentally the point of all this can’t be said more simply than – the open rate is inaccurate and measured inconsistently. For those reasons alone all industry participants should support an initiative designed to standardize the metric and ensure it is measured and communicated consistently.