CRE: Nielsen Diaries Worse Than You Thought

The Council for Research Excellence evaluated two primary factors affecting the accuracy of information gathered through ratings diaries — random errors and biases due to skewed audience samplings. Based on 11 years of Nielsen data, one study found that the margin of error in ratings derived from diaries is growing, meaning those figures are frequently off by more than 10%, long considered standard. The group’s other study found that ratings based on diaries continue to be skewed, despite efforts to reach a wider breadth of viewers by using address-based instead of phone-based audience samples.

Nielsen’s ratings diaries are becoming increasingly unreliable due to a proliferation of random errors. Separately, a lack of response from particular viewers such as young adults and Hispanics causes bias in sampling. These are according to two new studies from the Council for Research Excellence.

“You just can’t get the stability and the reliability with the current system that we have without a much larger sample,” says the CRE’s Richard Zackon.

The CRE released the findings in conjunction with the group’s Local Measurement Mini-Summit being held this afternoon in New York.

The CRE evaluated two primary factors affecting the accuracy of information gathered through ratings diaries — random errors and biases due to skewed audience samplings.

Based on 11 years of Nielsen data, one study found that the margin of error in ratings derived from diaries is growing, meaning those figures are frequently off by more than 10%, long considered standard. Total-day household ratings fall within that 10% just 11.3% of the time, the study found.

The ratings in primetime fall within that range 26% of the time. Ratings for weekday evening and late newscasts fall within the range 18.1% and 20.7% of the time, respectively, the study shows.

BRAND CONNECTIONS

Diary-based ratings measuring demographics are even more prone to error because audience samples are smaller, according to the CRE.

The group’s other study found that ratings based on diaries continue to be skewed, despite efforts to reach a wider breadth of viewers by using address-based instead of phone-based audience samples.

Univision’s Ceril Shagrin, who chairs the CRE Sample Quality Committee, says ratings diaries don’t represent a true cross-section of TV watchers because “those who don’t complete the diaries are very different from the ones who do.”

The study, for instance, found that diary respondents tend to be older than non-respondents, and frequently don’t have children living at home. Hispanics and young adults are among the demographics not accurately reflected in diary data, Shagrin says.

The research is in line with similar findings the CRE released in a 2009 study, and is part of the organization’s ongoing effort to work with Nielsen to improve the reliability of dairy-based data, she says.

“I am hoping that in our natural process we can determine what we learned and based on what was learned determine what could be changed,” she says. “There is more work to be done.”


Comments (11)

Leave a Reply

Gregg Palermo says:

October 8, 2013 at 8:31 am

Likewise, the 50% cooperation rate for the placement of PeopleMeters make me wonder if those who refuse boxes are not different than the half who accept them in their homes.

    Mark Gregory says:

    October 8, 2013 at 8:46 am

    Good point.

Michael Castengera says:

October 8, 2013 at 9:27 am

Nielsen? Inaccurate? I am SHOCKED!

Ellen Samrock says:

October 8, 2013 at 11:37 am

What I find strange is that here we are living in a world of FaceBook, Twitter and smart phone apps and yet Arbitron uses none of it. If you look at their website, you won’t find any use of this technology in their methodology of consumer (that is, viewer and listener) measurement. They use antiquated diaries, set top boxes and focus groups and cumbersome devices like the People Meter. It seems to me that they need to be using the tools that consumers, particularly younger consumers, currently use to communicate with each other to collect their data. One story has it that an Arbitron participant, for a laugh, strapped his People Meter onto his dog’s collar and wherever the dog roamed, that’s what the People Meter picked up. Conversely, I have never heard of a dog using Twitter.

    Ellen Samrock says:

    October 8, 2013 at 11:41 am

    Oh and before I get flamed, it should be noted that Arbitron and Nielsen are the same company.

ABELARDO BLANCO says:

October 8, 2013 at 12:31 pm

and we get to pay more every year for this cutting edge service!

Ellen Samrock says:

October 8, 2013 at 12:52 pm

Well, what do you know. According to this article in the NYT, Nielsen has just announced that it will start measuring Twitter chatter on TV shows. http://www.nytimes.com/2013/10/07/business/media/nielsen-to-measure-twitter-chatter-about-tv.html?ref=todayspaper&_r=0

    Wagner Pereira says:

    October 8, 2013 at 3:24 pm

    It was headline news on this website YESTERDAY if you had bother to read more and post less, blowing your comments posted earlier in the thread out of the water. And btw, since you apparently do not read very closely, Arbitron and Nielsen only became the same company the end of September, despite you inferring otherwise in your earlier post.

    Ellen Samrock says:

    October 8, 2013 at 7:53 pm

    Well, it isn’t mentioned on their website, it isn’t mentioned in this article and you’ve been too busy posting your worthless opinions and equally worthless venom against me to mention it. So I did.

Teri Green says:

October 8, 2013 at 11:14 pm

To be fair, Nielsen has never claimed to be scientific in it’s survey. It is not a survey of all people who watch TV. It is a survey of people likely to buy products. It caters to advertisers not to the population at large. This said, I am not disputing even for that it’s inaccurate, but I want to point out, too often people think Nielsen is a scientific survey of ALL people who watch TV. This has never been the case.

diane seghers says:

October 9, 2013 at 11:06 am

We have met the enemy and he is us. Our industry has attached an accuracy level to Nielsen ratings that even Nielsen says is unreasonable.


More News