Why Internet Measurement is Not Quite as Exact As We Thought
This particular post from Media Post’s “Online Metrics Insider” weekly email came across my desk, and it does an excellent job of summing up why the job of an Internet Research Analyst can be so darn difficult at times.
In theory, internet measurement should be the most exact form of audience measurement, especially when compared to the broad strokes estimates provided by Nielsen (Television), Arbitron (Radio), ABC Audit (newspapers), etc. It is far too easy for non-research geeks to look at internet audience numbers and assume they are 100% to the decimal point correct.
But as the post below illustrates, there are many, many different elements that go into developing an internet audience metric, all which affect the bottom line. Thus, while I have worked with Internet Analysts who like to report the internet figures down to the exact total (“We had 23,345,854 page views this month), I prefer to work with more broad terms (aka 23.3 million page views), accounting for the differences in measurement methodology.
The Numbers Just Don’t Add Up
by Jim Sterne , Friday, October 2, 2009
A funny thing happened on the way to the CMO’s office.
Between the realization of an eye-opening, game-changing insight gleaned from advertising test results and Web behavior data, the report you’re gleefully ferrying to the C-Suite wilted, turns brown at the edges and starts to dribble a slimy substance with a conspicuous stench.
The CMO immediately develops a nose-squint. The VP of Corporate Communications has that “Oooo, you’re in for it!” look in her eye and the VP of Advertising nudges the Director of Direct Marketing and says sotto-voce, “The golden boy is about to find out his day in the sun has turned him to toast.”
The CMO points to (but does not touch)
a traffic report from comScore
a traffic report from Hitwise
a chart from Compete.com
an ad banner report from Atlas
a traffic report from Omniture and
another from Google Analytics
“It’s like the old joke,” she said with no humor at all. “If you take all the economists in the world and line them up end-to-end, they all point different directions. What the hell is going on with these numbers? Are we getting thirty two and a half million people on our Web site or forty-four million?”
The first time you ran into this nest of nettles, you hopped over to the white board and cheerfully explained all about
multiple machine browsing
multiple browser browsing
multiple people on the same cookie
dynamic IP addressing
called pixel placement
You didn’t even get to the good stuff about comparing miles to gallons and how
different tools using
different date cut-off routines and
different methods to capture
different types of data to store in
different kinds of databases with a
different method of data cleansing and
different slicing and dicing segmentation to produce
different kinds of reports that ended up in
different feed for integration into
…before you were thanked for your help and shown the door — permanently.
You don’t fall for it this time.
This time you explain that the world of online marketing has been suffering from an delusion of precision and an expectation of exactitude.
You tell them that we live in a world of statistics and probabilities. We can’t count all the stars in the sky, so we don’t try. We don’t try to get an actual count of
bus poster readers
floor sticker readers
airline ticket jacket readers
sandwich board readers
Instead, we count some and estimate the rest.
You share the good news that we can do this better than any of the above — and we’ve got some astonishing tools and techniques for dynamically targeting the audience and optimizing each one’s experience.
You say, “We get 36.3 million people coming to our Web site.”
The CMO lowers her half-glasses and gives you the look you last saw when caught using the office copy machine for party invitations. So you add, “With a 4% margin of error and it’s a benchmark we can compare month over month from now on.”
“So somewhere between 34 and a half and 38 million,” she says.
“Pretty much right between them, in fact.”
Disparagingly, she asks, “You really can’t give me a more accurate number of how many people saw this digital marketing masterpiece that costs me tens of millions a year?”
“I can tell you whether our digital visitors are more engaged with our brand, come back more often, buy from us and discuss our products with their friends. How many people buy our products who saw our ads on CNN and ‘Oprah’ that cost you hundreds of millions a year?”
The VP of Advertising makes himself visibly smaller.
“I came here to show you a way that could save four million dollars of search marketing while boosting online sales by 6 to 8%,” you say.
The scowl leaves the CMO’s face. The odor of dubious data dissipates. Her eyes narrow as she leans forward and says, “Show me.”
The numbers don’t have to be precise — just compelling.
Entry filed under: Online.