Oxford University Press's
Academic Insights for the Thinking World

The polls aren’t skewed, media coverage is

The perceived failures of election forecasting in 2016 have caused many to suggest the polls are broken. However, scholars are quick to point out that more than polling failure this election has demonstrated that people have a hard time thinking probabilistically about election outcomes. Our research suggests skewed media coverage of polls may also be to blame: News media are likely to cover the most newsworthy polls, distorting viewers’ perception of the race.

Game of Polls

 News outlets are incentivized to cover polls because they are newsworthy. Poll coverage – much like stories on the horse race – are timely, novel, and conflict-ridden, meeting standards of newsworthiness. Viewers prefer the more exciting results, polls that show a close race or a candidate’s unexpected surge. Polls that show a consistent race, on the other hand, are less likely to get air time. The consequences of this cherry-picking is reflected in the nightly news, where coverage that emphasizes more exciting poll results may bias perceptions of candidate standings.

This selection process, or gatekeeping, affects the news we see and thus, represents a sort of media bias that can be difficult to study. This is in part because the only way to study this process is by examining the stories which make it past the proverbial chopping block, such as coverage of the 2008 US presidential election between Obama and McCain.

We collected data on primetime television poll coverage on major television networks (ABC, CBS, NBC, CNN, Fox News, and MSMBC) during the last five months of the 2008 presidential campaign, which suggested that media coverage reflects a different version of the race than actual polls.

When comparing the polls that made it to the air versus all polling data that was available for coverage, our findings show that news coverage reported on closer polls.

The graph below converts the reported margins between Obama and McCain in the 2008 US presidential election to standard-deviation units. The red line shows actual poll margins, and the blue line shows the poll margins covered on the news. At first glance, the two distributions are different, suggesting that media coverage of polls reflect a different version of the race than actual polls. Specifically, the actual poll distribution is more normally distributed whereas the coverage of polls distribution has a more dramatic spike clustered between -1 and +1. This suggests that news coverage reported on closer polls.

Comparing Poll Results to Poll Coverage in the 2008 Presidential Campaign
Comparing Poll Results to Poll Coverage in the 2008 Presidential Campaign by Kathleen Searles, Martha Humphries Ginn, and Jonathan Nickens. Used with permission.

More drama, more attention

Not only does media poll coverage vary from reality, but the coverage is not even accurately capturing the polls selected. In the 2008 election, polls suggested that Obama consistently held the lead over McCain.

In the graph below, the red line reflects the reported polls and shows that Obama consistently held the lead. The blue line shows which of these polls got the most air time. On average, viewers were more likely to hear about a close race than about the polls that showed Obama in the lead.

Distribution of Standardized Margins for Matched Polls and Poll Coverage
Distribution of Standardized Margins for Matched Polls and Poll Coverage by Kathleen Searles, Martha Humphries Ginn, and Jonathan Nickens. Used with permission.

When we look at the characteristics of the polls that do air on television, we find that changes in poll margins (as measured using a moving average of the last four margins) predict the total number of times a poll aired. In other words, significant changes in poll margins make it more likely that a poll will air.

For whom the poll airs

While much discussion following the 2016 Presidential Election has focused on the “broken” public opinion polling industry, there is a need for a discussion about how to properly cover polls. It may be helpful if news outlets were to decide upon a set of standards for how polls are covered, including qualifications for covering in-house polls versus outside poll firms, and share these standards with the public so as to make at least some of these processes less cloaked.

As the marketing value of polls has become more obvious, routines and norms of the newsroom are likely to be influenced. We suggest that a way to make sure these changes are positive is to reinvigorate discussion on better reporting of polls and increase the transparency of poll selection.

Our two pieces of advice: consider the source, and be wary of poll coverage.

Featured image credit: Politics & Pizza: Watching the U.S. Midterm Election Results by US Embassy Canada. CC-BY-2.0 via Flickr.

Recent Comments

There are currently no comments.