Open Rate? Render Rate? Do-You-Care Rate?

Suppose half of all doctors measure heart rate differently, and police departments calculate auto speed in multiple ways. Then, suppose the doctors and police use equipment that doesn't measure accurately.

I'm betting most of you would hope that these professions would move to ensure consistency and accuracy.

So, when Josh Baer, founder of this blog, recently questioned on Twitter the value and point of the EEC initiative to standardize and rename the open rate, I felt compelled to respond in more than 140 characters.

Here is a sampling of what Josh said:

"@LorenMcDonald while I agree with the logical argument, I think the whole thing is a waste of time. Not solving a problem. Open rate is fine."

"@LorenMcDonald when I say 'open rate' to ANYONE they know what I mean from the name. No one will know what 'Render Rate' means."

"@LorenMcDonald @Mostew who cares what the actual open rate is? How is reporting 1/3 less opens hurting me? I can still A/B split to compare."

Josh is not alone in his comments. But believe me, there are so many other things I'd rather talk about than the open rate, or, what I hope will become the recognized successor: the render rate. (Read feedback on the EEC blog and download the render rate document (PDF).)

Like the current irrational debate about single versus double opt-in, the core reasons that the EEC Measurement Accuracy Roundtable (which I co-chair) is recommending the render rate have gotten lost among the hyperbole and blog comments.

Why should you care? It starts with these essential points that are simple, straightforward and irrefutable:

  • Inaccurate: The open rate has become extremely inaccurate because disabled images, use of preview panes and HTML-unfriendly mobile devices lead to an underreporting of the true number of opens. Fellow eec Roundtable member Morgan Stewart has done analysis across several ExactTarget clients and estimates a typical underreporting of from 5% to 35%. Meaning a measured 30% open rate is actually from 31.5% to 40.5%.
  • Inconsistent: It's inconsistent, because different email service providers and marketing software providers calculate and report it differently. Among other things, some providers incorporate a click on a text message or HTML email where the tracking image has not loaded in their open rate calculations, and many others don't.
  • Misused: It's being used to assess how well a message performed. The open rate, even putting aside its inaccuracies, does not tell you how well an email performed. It does tell you how well the combination of your "from" name, subject line, brand reputation and potentially preview-pane copy/images motivated recipients to open their email.

Oops! We don't even know if it measures that, because the email might load in the preview pane without the recipient viewing it.

Open Rate Measures Process, Not Output

Only a metric that measures whether the campaign met business goals, such as total revenue, revenue per email or conversions, can accurately reflect success or failure.

The open rate is a process metric, meaning it measures actions on the email. It's not an output metric, which measures business goals, but many marketers treat it like one.

The open rate can help assess process-related problems with your emails. But, it won't tell you whether your email delivered more revenue, leads, demos, or downloads, helped retain clients or collected more referrals.

I've met thousands of email marketers over the last several years, but not a single one has told me he or she would get a raise or bonus by increasing open rates.

I'll also agree that the open rate has its uses:

You want to use it to measure a branding campaign instead of a revenue-generator? That's fine. Newsletter publishers want to measure whether subscribers are reading their articles? OK.

Or, you are a combo brick-and-mortar-plus-online retailer such as Borders or REI that uses opens to help measure reach and the impact of email in driving people to a physical store? Got it.

Why Should You Care?

Josh, you and others should care about redefining and renaming the open rate because the average marketer still doesn't understand what an open rate is or actually measures.

In expert hands, the open rate is still inaccurate but not harmful. The metric is worrisome, however, when uninformed marketers communicate the success of their last campaign or newsletter to their management based on a few percentage-points' increase in their open rate.

If we are going to move the email industry forward – away from the batch-and-blast, just-send-more, "deliverability is someone else's responsibility" attitude of many – then ground zero starts with redefining the open rate.

Talking about the open rate might be a bore and waste of time, just like talking about blocked images, blacklists, authentication protocols and bounce rates are a poor use of a marketer's intellectual capital.

We should be talking about integrating email with other channels, lifetime customer value, cool uses of dynamic content, kick-ass trigger-email approaches and the like.

So, why are we talking about redefining the open rate? Because, until we can move the industry beyond talking about open rates as a key benchmark, we likely won't be able to talk about those higher-level matters.

Putting aside philosophical arguments about the role and value of the open rate, fundamentally the point of all this can’t be said more simply than – the open rate is inaccurate and measured inconsistently. For those reasons alone all industry participants should support an initiative designed to standardize the metric and ensure it is measured and communicated consistently.

If you haven't already, please download the EEC render rate document (PDF) and then post your comments on the EEC blog.

5 Responses to “Open Rate? Render Rate? Do-You-Care Rate?”

  1. Derek Harding
    March 9, 2009 at 5:04 pm #

    While I'm with you on redefining open rate to a consistent term I'm concerned that it's still a derivative of deliveries and delivery has not yet been defined.

    Derek

  2. Loren McDonald
    March 9, 2009 at 6:31 pm #

    Derek,

    Appreciate the support and feedback. We hear you on the "delivered" metric (here and with your comments on the EEC blog).

    Actually, the original EMAC group (which was absorbed into the EEC and become the Measurement Accuracy Roundtable) did define the "delivered" denominator upon which open, click-through and other rates are calculated.

    Unfortunately the delivered metric was never communicated widely. We discussed this on our most recent MA Roundtable call and will begin an initiative to communicate it to the industry.

    Greatly appreciate the feedback and persistence! Can we draft you into becoming a member of the Roundtable?

    Loren
    Silverpop/EEC Measurement Accuracy Roundtable Co-Chair

  3. Mitch Tarr
    March 9, 2009 at 10:48 pm #

    I think the real value of this article lies in the interest in moving past open rate (not to uniform render rate) but to bottom line. Campaign goals met vs not. Hopefully campaign goals can be measured by something more meaty that opens and clicks.

  4. Loren McDonald
    March 10, 2009 at 1:51 am #

    Mitch,

    Thanks for the comments. I couldn't agree more. As an industry we need to focus our conversations around achieving business goals and less on the latest average this and that rate.

    That being said, as a first step we do need to define and standardize our most basic performance metrics.

    Loren

  5. Calman
    January 12, 2011 at 4:07 am #

    Anyone know how Vertical Response calculates open rate? Are they in compliance with the EEC's recommendation for render rate?

UA-9835597-1