The shadow that stalks the numbers of the Democrats: can we trust the polls?


Over the past two months, FiveThirtyEight’s congressional polling average has shifted from a two-point Republican advantage to a half-point Democratic advantage. His approval poll average for President Biden went from Biden underwater by 17 points (meaning his “disapproval” percentage is 17 points higher than his “approval”) to 11 more modest points. Speaking a little more figuratively, Democrats have generally moved from a position of panicked hyperventilation to a position of cautious apnea.

But then there’s this shadow that follows the party like a rain cloud: What if the polls were wrong? Or, really: What if the polls were wrong? Again?

Sign up for How To Read This Chart, a weekly data bulletin from Philip Bump

When Republicans topped the polls in 2014, few people noticed. Republicans held an advantage in generic ballot polls (poll questions that pit an anonymous Republican against an anonymous Democrat) and things just turned out worse than expected. Two years ago it was Barack Obama who beat the polls, you know. You gain some, you lose some.

Then came 2016. Hillary Clinton was leading in state polls and polling averages. The models gave him a strong chance of winning. Democrats were confident. But many people failed to notice how the patterns and polls changed in the final days of the election. Then came election day and, well, you don’t need me to explain what happened.

In 2018, things worked out again for Democrats, and it looked like the ghost may have been exorcised. The Democrats were expected to do well, and then the Democrats did well.

Then came 2020. Joe Biden’s lead was wider than Clinton’s and observers (ahem) noted that even a Clinton-level failure in state polls wouldn’t keep him out of the House. White. Then there was a near-Clinton-level failure, and Biden fought his way to the presidency.

So we come to 2022. Will it be like 2018, when Trump wasn’t on the ballot and the polls were about the money – with Democrats wanting to retain a majority in the House? Or will it be like 2014 and 2016 and 2020 – worse than expected?

This Trump question on the ballot has been the way I’ve looked at things for some time now. Then I saw a tweet from G. Elliott Morris of The Economist (who recently published a book on polls) that underscored an important point: while the National Poll by Generic Ballot reached its goal in 2018, the polls at the state level were still off the mark. .

I decided to put this to the test. I pulled state-level polling averages for Senate contests from FiveThirtyEight and compared them to the actual voting results. The trend is pretty clear: Senate poll averages consistently underweight Republican performance.

How to read this table: At the top are bee plots positioning each Senate contest based on the distance between its predicted results (the average of the polls) and the actual results. States in which Democrats beat the polls are in blue; states where Republicans have done so are in red. States that switched from 2016 to 2020 are in bold.

At the bottom is an admittedly complicated comparison of the poll averages to the results. The average polling margin is shown from top to bottom, with a greater Democratic advantage at the top and a greater Republican advantage at the bottom. From left to right, the actual margin, with Democratic wins on the left of the center line and Republicans on the right. The key is the diagonal: States to the right of it saw Republicans outperform the average, and states to the left saw Democrats outperform.

It depends on the FiveThirtyEight average, but since it’s one of the tools people use to gauge how things are going, I feel comfortable using it as a metric. (If you’re curious, I’m using the average that takes into account non-poll factors, like fundraising.) And what we’re seeing is a pattern of those polling averages underestimating how Republicans would get away with it.

In a tightly divided Senate, the possibility of the polls being shifted again, even slightly, means a change in control of the chamber, as Morris Noted on Twitter.

“If you think today’s Senate polls have no national bias toward Democrats, then they’re about 80% in a majority (based on polls alone, not fundamentals),” he wrote. “If you think the polls will be as skewed as they were in 2020, they are slight underdogs.”

In other words, just enough races will go the other way to give the Republicans a slight edge. It’s easy to see where this could happen. In Pennsylvania, FiveThirtyEight lifted Democrat John Fetterman about five points. In 2020, he estimated Biden would win the state by about five points. He won by just over 1 point. In 2018, however, Democrats outperformed by nearly two points in the Senate race. And in 2016 they belowperformed by a slightly larger margin.

(Let me note that this is not meant to disparage FiveThirtyEight! It just has the misfortune to have objective poll data that it presents in a usable format, which I appreciate.)

Of course, there is another factor at play: the passage of time. We are still about 70 days away from the election in which a lot can happen. This month, I looked at how the generic national ballot has changed over the past four midterm election cycles over the past 90 days. In two the Democrats improved and in two the Republicans did. Importantly, during the two years when the national average was far from the actual results, the shift was against the Democrats.

This is a small sample! And I confess that I don’t have a good answer to the question of what we can expect. It’s a weird year with weird dynamics (a hyperpresent and unpopular former president) and huge political changes (the overthrow of Roe vs. Wade) that have an effect.

But I can say this: that shadow surrounding the Democratic polling numbers is, indeed, something that could rightly send shivers down your spine.


Not all news on the site expresses the point of view of the site, but we transmit this news automatically and translate it through programmatic technology on the site and not from a human editor.
Back to top button