When President Donald Trump held a campaign rally at Lancaster Airport last month, he told the more than 10,000 people in the audience that the pollsters were wrong about his standing in the presidential race.
“They’ve got themselves a big, big surprise coming, don’t they?” Trump said. “It’s like four years ago.”
It turns out part of Trump’s prediction was right: like four years ago, the 2020 presidential campaign came down to a razor-thin margin despite polls predicting a comfortable win in Pennsylvania for Democrat Joe Biden.
Trump won Pennsylvania by 44,000 votes four years ago. Though vote counting continues from last week’s election, Biden appears set to carry the state by 50,000 votes.
LNP | LancasterOnline spoke with Berwood Yost, the director of the Franklin & Marshall College Poll. The poll he manages consistently showed Biden leading Trump since last spring by 6 to 10 points. (You can read the College's last poll before the election here.)
The conversation has been edited for brevity and clarity.
Joe Biden won, but just barely. How does that affect your assessment of the F&M Poll’s performance?
I get all the hand-wringing about polls and polling. Everybody wants to point to the polls, and talk about them being off. I think that’s all fair, but focusing on just a lead at a given point in time -- which is all these polls can really provide -- is sort of an overestimation or overreliance on something that is probably the least important information in these polls.
Polls are limited in what they’re able to do. For instance, our poll was done basically two weeks before the election. We weren’t trying to produce an estimate that said, ‘This is what’s going to happen.’ We were trying to paint a picture of the race at the time the survey was conducted. We don’t look at just one thing when we do that kind of assessment. We look at what are the issues people care about? How are they feeling about their personal finances? How do they feel about the candidate personally?
Depending on how people are judging the race, they might go one way or another if you look at the different indicators. If you’re someone concerned about the economy, economic performance is important. Trump had an advantage in economic concerns, but concerns about coronavirus probably lessened his chances.
I think it’s a far better use of poling to understand the whole picture. From that perspective, the poll did its job. Most notably, it showed the fact that Vice President Biden was doing better than Hillary Clinton in counties she had won in 2016.
So are pollsters doing a bad job of explaining their results to the public, or is the public not understanding what a poll can tell us?
As pollsters, we are constantly thinking about our work to make it better. I would never say we shouldn’t change. We should always change, we should always be thinking about how we could improve.
But one of the ways we can improve is by doing a better job of reporting the limitations of polls. Instead of being focused on the horse race between two candidates, we should be encouraging people to spend more time thinking about multiple indicators that might reflect how people will behave.
I think it’s important to think about what a special case election polling is compared to other kinds of polling. For example, assume we are trying to understand the state of health in Lancaster County by conducting a community health needs assessment. First of all, we know who those people are. It’s easy to define people who live in Lancaster County. We know what the residents look like by census characteristics. We have a really clear and defined population.
A health poll like that might find that 12% of resident smoke, 20% binge drink, 30% are overweight, 18% have anxiety. Public health officials would then look at all of those things collectively to assess the health of a community. You should do the same when you think about the state of a race or a campaign.
We don’t have such a thing in these polls. We’re trying to predict who a likely voter is. Pollsters try to create a sample of a population that will exist for one day. Sometimes, I think it’s amazing the polls are in the ballpark at all, when you’re trying to define a population in only a very limited time frame.
What went wrong that led the Franklin & Marshall College poll to be several points off from the actual results?
Every election is different. Turnout was as high as it’s been in 100 years. We don’t really have good models for turnout that’s this high, that’s one of the things we’re going to have to look at.
We’re going to have to look at what a really high turnout election means. More Republicans voted than we expected. The votes aren’t done being counted in the state. We still don’t know what the final margins are. Everybody wants to rush to judgment.
What surprised you in this election?
It’s clear that it looked like the conditions were set for an advantage for Democrats and that didn’t materialize. That’s going to be interesting to explore.
I think the most important question we’ll have to understand is how did the electorate look compared to how we expected it to look. In 2016, a couple things became apparent: our samples underrepresented white working class voters, in which education plays a role for voter preference, along with the fact that a lot of people changed their preferences in the last week or so of that election.
In 2016, the polls kinda moved up and down. Hillary Clinton had an 8-point lead, then it was 2 points ahead for Trump. Over the course of the campaign there was a lot of variability. That wasn’t the case this year -- they stayed pretty consistent.
Should reporters change the way they report on polls?
We should all be working together. I’m not sure that there should be a rush to judgment about the polls. I know it’s newsworthy, but we just have to set clear expectations of what the polls are telling us. For too many people, it all boils down to the polling averages, which hide the variability that is inherent in every poll.
If you were to look at our final poll, which showed Biden at 50 and Trump at 44, the gap between the candidates was 6 points. But if you were to calculate the margins of error around that, anything from a 4-point Trump win to a 14-point Biden win would’ve fallen within that margin of error.
That’s the problem: polls are variables, and when you’re looking at two estimates of a poll, you’re talking about twice the sample error. When we start adding all the polls together in an aggregation site [like FiveThirtyEight or Real Clear Politics], we miss that.
In 2016, there were a lot of hypotheses that Trump supporters were not inclined to reveal their preference when contacted for polls. Do you think that continued during this election?
It’s hard to find evidence for it. What’s more likely is that there may be Trump voters who won’t even pick up a phone.
Certainly it’s a combination of things, but if we keep our eye on the big picture, the polls told a story of an incumbent president. Depending on how people judged him, it was always going to be competitive. His job approval rating was high enough where he was always in the game. It wasn’t so high he was going to win in a walk or wasn’t so low like Tom Corbett’s after his first term.
Were there any indicators for the poor Democratic performance down ballot?
Here’s the thing, no Democratic congressional incumbents in Pennsylvania lost.
The thing that’s clear is we’re seeing some regional reshuffling of the vote in Pennsylvania between urban/suburban versus rural areas. Those things are real, and you can see them happening throughout the state.
I wonder how much any of the campaign messages below the presidential race were being heard. The volume of spending in the presidential campaign was massive, and the state House and state Senate candidates spent smaller amounts, comparatively, advertising their campaigns. It’s hard to know what messages got through
We also had a new ballot that didn’t allow for straight-ticket voting. I don’t know what effect that might’ve had. It’s pretty clear that Republicans were able to get out and vote in large proportions, and that’s something we’re looking to try and learn more about, as well as finding out what the electorate actually looked like.
Anything else you’d like to add?
I just hope as we go forward, we can all do better talking about these polls. That’s my hope.
We promise to look and figure out how we can keep improving, given the limitations of election polling. Hopefully, journalists will be with us to help talk about the limitations, the variability and the possibilities in polling.