Are you suffering from Metric Myopia?

Imported from the Blogactiv.eu blogging platform, closed without warning in 2021. Links, images and embeds not guaranteed, and comments not displayed.

Before you obsess over your next analytics report, read the recent article by Ev Williams (Twitter and Medium cofounder) on measuring what matters.

William's article was prompted by the infantile way otherwise supposedly intelligent people (venture capitalists, Wall St analysts, tech & business journalists...) measure the value of companies like Twitter and Instagram:

we literally say one company or service is “bigger” based on a single number - specifically, number of people who have “used” it in the last 30 days. Even without getting into how “use” is defined, this is dumb. - A mile wide, an inch deep

Similarly, some websites are apparently 'bigger' than others because they have higher "Unique Visitors", measured by (the very flawed) Comscore. Of course, not all people suffer from what I'm going to call Metric Myopia, and can actually think in more than one dimension at a time:

William's article provides a short tour of the debate around measuring time spent reading, triggered by Chartbeat’s Tony Haile “Attention Web” article, which included this data among others.

"So by all means measure social sharing - it'll help you figure out how to create content that gets shared! But don't use that data as a proxy for "content that gets read". There is no correlation" - You're probably analysing the wrong thing (my take on Tony Haile's data and analysis) But even when you use meaningful metrics like Total Time Reading, Willaims points out, it's still just a proxy for the actual goal:

As the writer of this piece... I want to make an impression on how you think.

Sufferers of Metric Myopia have forgotten that. In fact, they're so focused on their 'golden metric' that they've not only forgotten it's just a proxy for their goal, they've forgotten that it's probably a crap proxy, indicating nothing useful. The metric replaces the goal.

Changing minds, or just counting clicks?

If Metric Myopia leads investors to make simplistic bets based on single, unrepresentative proxies for value, and forces publishers to blindly optimise clickbait strategies to meet advertisers' myopic click obsession, what are the symptoms in government communicators?

Are we being understood? Are we changing minds?

After all, getting clicks for public communicators is not a question of survival, as it is in publishing. Nevertheless, I see the same obsession: are we getting views? are we getting shares? But rarely: Are we being understood? Are we changing minds?

When defining metrics, government communicators should not blindly adopt the 1990s-era metrics which led news media to the brink of destruction: there's no point getting lots of content views and shares if that content doesn't help you reach your goals. Moreover, government communicators' goals do not involve selling stuff, so why are you using advertising-oriented metrics?

Now measuring influence isn't easy, but instead of rehashing my 2011 rant What is influence? or, Why I don’t care about my Klout score, I'll simply leave the last word to Ev:

If what you care about ... is impact on the world, it all gets very slippery... Different services create value in different ways. Trust your gut as much (or more) than the numbers. Figure out what matters and build something good.

Amen. PS. Sachin Kamdar wrote a nice response to Ev's post where he agrees but points out that Ev's conclusion, above, is "about as sexy and helpful as ending an article on weight loss by reminding the reader to “Burn more calories than you consume"". Although it contains a huge plug for his company, it's worth a look. As for what I would define as "something good", try Being Useful beats Being Tuneful.

----

Further Reading

• A mile wide, an inch deep, by Ev Williams
• Taking on the real problem with metrics
• What is influence? or, Why I don’t care about my Klout score (2011)
• Being Useful beats Being Tuneful (2011)
• Resources tagged metrics (8), monitoring (8) and media (124) on my TumblrHub