fourth estate

Opinion | Journalists Are Hooked on Polls and They’ll Stay That Way

The polls might be wrong sometimes but the solution isn’t doing away with them, just more transparency about their limits.

This Dec. 11, 2012, file photo shows a 1947 survey for the Gallup Poll at the University of Iowa library in Iowa City, Iowa. The document is part of the personal papers of the late George Gallup Sr., that show the development of nationwide public opinion polling. A new survey highlights the two-front problem facing the polling industry: People don’t trust polls, and many say they are unlikely to want to participate in them. Gallup one of the most respected names have been conducting experiments to refine their likely voter models after the 2012 election seemed to foil processes that had worked well in the past.

Inside the temple of political polling, the high priests are starting to lose their faith.

The foundational organization of political polling, Gallup, bailed on presidential political polling in 2015 after having made a botch of the 2012 election cycle. Citing a change in their research focus, the Pew Research Center dropped out of the horse-race business in 2016. But no political polling insider has served himself such a heaping bowl of mea culpa as did Monmouth University Polling Institute director Patrick Murray last week in a Newark Star-Ledger op-ed.

“I blew it,” Murray wrote. “The final Monmouth University Poll margin did not provide an accurate picture of the state of the governor’s race.”

Murray didn’t announce his departure from the business, but he walked right up to the line. His poll had reported a double-digit lead by Democratic Gov. Phil Murphy over Republican challenger Jack Ciattarelli in the final days of the campaign. That proved wrong when Murphy won in more of a squeaker than a landslide. Murray’s remorse appeared genuine as he speculated that his poll might have stymied Ciattarelli’s fundraising and voter mobilization and skewed the election.

But Murray’s apology isn’t a one-off. The veteran pollster, whose outfit has earned an “A” from polling maven FiveThirtyEight’s Nate Silver, writes that if misses like his New Jersey call are not anomalies, pollsters need to reconsider whether they have any business releasing late-breaking polls. The idea that pollsters might be doing more harm than good is an idea that you expect to hear from polling critics, not a professional pollster. If Murray’s fears are confirmed, should journalists even bother citing polls in their coverage?

Obviously, pollsters aren’t about to throw themselves down a well even though the scorebook reveals how inaccurate their polls can be (Truman vs. Dewey in 1948; Reagan vs. Carter in 1980; Trump vs. Clinton in 2016; and many, many more). But Murray’s misgivings and the exits of Gallup and Pew invite us to ask what political reporting might look like if they did. How much would campaign news change? Have reporters grown so dependent on polls that they can’t cover campaigns without them? And finally, is Murray on to something when he asks if the pollsters’ spotty performances have fed a cynical backlash against journalism?

The press, the public and academicians have been asking Murrayesque questions about election polls for decades, as W. Joseph Campbell writes in his comprehensive 2020 book on the subject, Lost In a Gallup: Polling Failure in U.S. Presidential Elections. Journalists from James Reston to Eric Sevareid to Mike Royko to Nicholas von Hoffman to Dan Rather to Ariana Huffington have damned the polls as a nasty influence on elections and pointed out their deficiencies. It’s beyond the scope and ambition of this column to assess the accuracy of polls, so we will seek a more modest quarry: How would the departure of pre-election polls, or a refusal by journalists to use them, change coverage?

To be a journalist is to be a soothsayer, especially for political journalists, who speculate on everything from who stands to win the next election to what the chances are that a bill might pass to what the president might do next. Pollsters and political journalists use poll data to divine the future the way Doppler radar and satellite imagery is used for tomorrow’s weather report. You might find that comparison a stretch but both polling data and Doppler radar are nuggets of data used to predict future events. Neither polling predictions nor weather forecasts have ever been considered bulletproof. The same goes for stock predictions and sports odds. They’re born flawed.

The mind resists imagining the world of journalism without polls because polls are journalism and have been since 1824, when reporters cited the results of non-scientific straw polls at public political gatherings. The criticisms of polls is valid. They’re no way near as “scientific” as their proprietors would like you to think they are, but how much worse are they than the much vaunted “shoe-leather reporting” from the campaign beat that attempts to record the voters’ temperature? Shoe-leather reporters have been known to misjudge the electorate’s mood, too, and nobody calls for their disbarment.

The banishment of polls would obviously complicate the lives of assigning editors. Poll-ranking has never been the sole measure of who deserves ink, nor should it be. But the editor’s job is to apportion coverage. Polls — as imperfect as they are — give the public and the candidates some inkling of the electorate’s thinking. Few editors assign campaign stories based on a candidate’s poll-ranking alone. If they had during the 2020 primaries, Kamala Harris, who performed so poorly in the polls, wouldn’t have gotten the expansive coverage she did before dropping out. Did editors do readers a favor with all that Harris coverage? To reward an also-ran with disproportionate attention and neglect the more popular candidates would seem to neglect a publication’s news mission.

If editors declared political polls as garbage and refused to use them in reporting, how would they decide which candidates to cover? Perhaps they’d eyeball the size and enthusiasm of rallygoers; assess the worthiness of their campaign promises; compare campaign donations; yardstick the number and heft of endorsements; count the number of candidate yard signs and bumper stickers; and consult other journalists’ coverage. By the time they had finished their consultations, they would have almost reinvented polling by inferior means. Without the occasional reality check of polls, would campaign reporters find themselves indulging in groupthink and become prisoners of access? It doesn’t take much to imagine a campaign reporter writing a Murrayesque self-criticism when his poll-less coverage turns out to misjudge a candidate’s support. The closer you get to banning pollsters the more you come to admire them.

A world without polls might force reporters to become more creative. Any novice journalist can turn a political poll into a story by comparing it to a previous poll and using that as a platform to explain why a candidate is up or down in the polls, how he has gained or lost momentum, why he should end his campaign, and so on. (Journalists have been known to herald a swing of one or two points in a poll, which is bogus because such a small change is inside the margin of error of most polls. In such cases, it’s equally likely that neither candidate has risen or fallen.) Denied the poll crutch — and it can be a crutch — reporters would file more coverage about the candidates’ stands on issues or devote themselves to descriptions of the campaign and biographical accounts. Such a shift will produce a sense of well-being in the good government liberals who believe campaign coverage should limit itself to coverage of the white paper “issues.” But who would read steaming vats of such stuff if that’s all journalists produced? Reader interest in issues is not absolute. They also want to know who has a chance of winning. Without polls, it would be harder for editors to tell a contender from a long shot and it might bestow upon those editors power they really shouldn’t have.

You can make a good case, as Murray does, that inaccurate polls can drive voter cynicism. But any honest appraisal would have to include a study of what sort of cynicism that poll-free reporting might produce. It would be a mistake to instill our exclusive faith in the subjective assessments of campaign reporters and their assigning editors. Just because polls occasionally miss the mark shouldn’t make journalists the new oracles of Delphi. Voter cynicism might not be a product of polling as much as it’s a product of politics.

If polls do breed cynicism, one way to relieve that cynicism would be for pollsters to engage in more of the Maoistic self-criticism found in Murray’s op-ed. It’s probably too much to expect pollsters to stop polling a month before elections to allow the voters’ palates to clear by election time. And forget about passing a law that bans them — you know, the First Amendment. But perhaps one way for pollsters to reclaim credibility for their product would be for pollsters (and the journalists who cite them) to frame their polls inside a consumer-product advisory that confessed frequent fallibility. It would also help if pollsters stood up to take their lumps the way Murray did. Likewise, independent ratings of polls, such as this one by Nate Silver, would bring some needed accountability to the profession. Journalists should educate readers on the limits of polling by showcasing comments from their critics. So when Silver writes as he did in the spring, that “It was a mediocre year for the polls in 2020” and “Polls in 2020 considerably overestimated Democrats,” that deserves billboard coverage. When polls don’t reflect reality, pollsters mustn’t make excuses like “people don’t answer the phones anymore” or “we should have polled closer to Election Day” or “voters have become shy about stating their true preferences” or our modeling was not perfect, as Murray conceded. If a poll fails, it’s the pollster’s fault, and he should admit it.

In other words, we need more self-critical pollsters like Patrick Murray. It would be a pity if, having gone this far in his assessment of his own work, he retired from election polling.

******

Here’s Time magazine going all Maoist on journalists in 1948 when its reporting was as bad as anything the pollsters turned in during the Truman vs. Dewey campaign. “[The press] was guilty of laziness and wishful thinking: it had failed to do its own doorbell-ringing and bush-beating; it had delegated its journalist’s job to the pollsters.” Criticize yourself in an email to [email protected]. You don’t want to know my email alerts margin of error. My Twitter feed lies to pollsters. My RSS feed once killed a telephone pollster with a curse.