CRE: Nielsen Diaries Worse Than You Thought

Nielsen’s ratings diaries are becoming increasingly unreliable due to a proliferation of random errors. Separately, a lack of response from particular viewers such as young adults and Hispanics causes bias in sampling. These are according to two new studies from the Council for Research Excellence.

“You just can’t get the stability and the reliability with the current system that we have without a much larger sample,” says the CRE’s Richard Zackon.

The CRE released the findings in conjunction with the group’s Local Measurement Mini-Summit being held this afternoon in New York.

The CRE evaluated two primary factors affecting the accuracy of information gathered through ratings diaries — random errors and biases due to skewed audience samplings.

Based on 11 years of Nielsen data, one study found that the margin of error in ratings derived from diaries is growing, meaning those figures are frequently off by more than 10%, long considered standard. Total-day household ratings fall within that 10% just 11.3% of the time, the study found.

The ratings in primetime fall within that range 26% of the time. Ratings for weekday evening and late newscasts fall within the range 18.1% and 20.7% of the time, respectively, the study shows.

Diary-based ratings measuring demographics are even more prone to error because audience samples are smaller, according to the CRE.

The group’s other study found that ratings based on diaries continue to be skewed, despite efforts to reach a wider breadth of viewers by using address-based instead of phone-based audience samples.

Univision’s Ceril Shagrin, who chairs the CRE Sample Quality Committee, says ratings diaries don’t represent a true cross-section of TV watchers because “those who don’t complete the diaries are very different from the ones who do.”

The study, for instance, found that diary respondents tend to be older than non-respondents, and frequently don’t have children living at home. Hispanics and young adults are among the demographics not accurately reflected in diary data, Shagrin says.

The research is in line with similar findings the CRE released in a 2009 study, and is part of the organization’s ongoing effort to work with Nielsen to improve the reliability of dairy-based data, she says.

“I am hoping that in our natural process we can determine what we learned and based on what was learned determine what could be changed,” she says. “There is more work to be done.”