Are presidential polls skewed? Not much evidence
  • Berwood A. Yost

By BERWOOD A. YOST
Published Oct 28, 2012 00:02

"There has been no shortage of targets for partisans of all persuasions this election season. But one group has come under fire from all sides: pollsters, who in these polarized times have become the political equivalent of lawyers."

Jim Rutenberg's characterization of polls and pollsters in The New York Times sounds familiar. He is describing the frequently repeated criticism that public polls have too many voters from one party, and too few from another, represented in their samples. But you may be surprised to learn that Rutenberg was writing about the 2004 presidential election and that it was the Democrats who were complaining.

The refrain is a familiar one. Complaints and criticisms about polling and pollsters appear in every election and it always comes from the same source: supporters of the candidate who is trailing. But is there any evidence, besides the complaints of aggrieved partisans, that the polls are skewed?

Not much!

In fact, public polls have a strong record of accurately predicting election outcomes. In 2000, 2004 and 2008, the national polls missed the final results by only about one percent according to the National Council on Public Polls. Between 1960 and 2000, the average miss was by less than two percentage points. The state polls fared similarly in 2004 and 2008, missing by less than two percentage points in both years. These "misses" are well within the error expected when using a sample to represent the intentions of voters.

But public polls could still be biased if they consistently under-represented the performance of one party's candidate even if they have a good record of predicting election outcomes. So, have public polls consistently favored one party's candidates over another?

There is strong evidence that they have not. A recent analysis of public polls showed that the Democratic presidential candidate's vote total was overestimated in five presidential election years between 1972 and 2008 while the Republican presidential candidate's vote total was overestimated in four elections. Apparently, public polls have no record of favoring one party's candidates over another's.

Overall, history shows that public polls perform well and without consistent bias. History also shows that partisans will nevertheless complain and criticize the polls.

The most common complaint in 2012 has been that Republicans are under-represented in samples of likely voters. Partisans believe survey samples should contain a specific percent of Republicans or Democrats based on previous elections, but most public pollsters do not adjust their likely voter samples to reflect some pre-determined number of Republicans or Democrats.

Instead, pollsters calculate who is likely to vote by asking questions about interest in the election, likelihood of voting, and past voting behavior. Survey participants define their own interest in the election and consequently reveal their own likelihood of voting.

A public pollster's sample typically does not rely at all on what turnout looked like in previous elections. The party composition of a survey sample is determined by what voters' plan to do and not on what happened in the last election. The simple truth is that voters' intentions to vote do not always match the voter turnout partisans want and expect.

Finally, we should remember that public pollsters are using methods today they have crafted over 60 years to produce accurate, unbiased estimates about the presidential race. What possible incentive does this industry have to produce skewed results when, in fact, the credibility of the entire profession relies on getting the correct results? Pollsters would only harm themselves if they manipulated polling results to reflect their personal political beliefs.

Informed citizens should remain skeptical of the polls they see. Not every poll is designed and executed with the same care or skill, and any one poll can be seriously wrong due to sample error, poor questions or faulty interviewing.

Reputable firms will release the details of their polls so that consumers can judge the quality of the polling for themselves. Armed with these disclosures, voters are smart enough to spot those flaws if they look closely. So, while skepticism of individual polls is healthy, the condemnation of an entire, diverse industry with a strong track record is just more partisan politics.

Taken together, public polls are unlikely to mislead us. Unfortunately, the same cannot be said about the partisans who criticize them.

Berwood A. Yost is director of the Floyd Institute for Public Policy, Center for Opinion Research, at Franklin & Marshall Collge.

blog comments powered by Disqus
Switch to Full Site
Download our Apps
Tablet Zoom Control: Zoom | Normal