There’s been so much news going on that I nearly forgot about the existence of a Supreme Court vacancy when participating in a virtual conference this morning. And in some ways, the news makes life hard for us polling analysts. We have had, in rapid succession: the death of Justice Ruth Bader Ginsburg, a series of threats by President Trump in which he refused to commit to a peaceful transition of power, the announcement of Trump’s intention to nominate Amy Coney Barrett to replace Ginsburg, the release of substantial information about the president’s tax returns by the New York Times, the first presidential debate, the news of Trump’s COVID-19 diagnosis and subsequent hospitalization — along with a cluster of other cases at the White House — and now his plans to leave Walter Reed Medical Center later today.
With that said, the debate provides a clear enough line in the sand — with many pollsters planning releases just prior to or just after the debate — that it’s worth doing a before-and-after comparison of the polls.1 One way to discern this is via our national polling average, where Biden’s lead has inched up to 8.1 points from 7.1 points before the debate:
Another way to see this is by comparing post-debate polls to the last, pre-debate version of both national polls and polls of the same states by the same polling firms. That’s what I’ve done in the table below, covering the 11 national polls and 11 state polls that have been released since the debate. And on average, Biden gained 1.5 percentage points from the pre-debate to the post-debate versions of these polls.
|Pre-debate poll||Post-debate poll|
|Pollster||End date||Biden lead or deficit||end date||Biden lead or deficit||Diff.|
|MI||Public Policy Polling||8/29||+4.0||10/1||+6.0||+2.0|
|US||John Zogby Strategies||8/29||+4.5||10/2||+2.0||-2.5|
|US||Data for Progress||9/15||+11.0||10/1||+10.0||-1.0|
In some cases, the pre-debate poll was conducted months before the debate. But even if we only focus on the polls that released results at some point in September, the top line is similar: Biden’s gain is 1.4 percentage points, on average. And for what it’s worth, a swing of 1 or 2 points toward Biden is consistent with the effects that first debates have had for challengers in the past — although we should note that sometimes a debate bounce fades after a couple of weeks.
These averages, of course, conceal a lot of variation within individual polls; depending on which poll you look at, Biden’s national lead is as large as 14 points (in the most recent NBC News/Wall Street Journal poll) or as small as 2 points (in a John Zogby Strategies poll). I’ll admit to finding the NBC/WSJ poll intriguing, as it’s the only live-caller national poll we have since the debate, and NBC/WSJ polls tend to stick pretty close to our polling averages. Still, averaging the polls is the right strategy in the long run.
Because so many polls have recently been released, you’re going to have a few eccentricities. Remember, an 800-person poll nominally has a 3.5-point margin of error. However, that margin pertains only to one candidate’s vote share; in a two-candidate race, the margin of error on the difference between the candidates will be roughly twice that then, or 7 points in this case. That means, if Biden is truly ahead by 8 points, it should be fairly routine to get everything from a 15-point Biden lead to a 1-point lead. And keep in mind that with a confidence interval of 95 percent, the margin of error only covers 95 percent of polls; 1 in 20 polls should fall outside that range. (At least in theory; in practice, pollsters are often reluctant to publish polls that appear to be large outliers.)
It’s harder to tell, though, what effect the president’s COVID-19 diagnosis may have had on the race. YouGov conducted polls right before and after the news and found Biden ahead by 8 points in both. Morning Consult shows Biden having gained 1 point over the past few days. And Ipsos had Biden ahead by 10 points in a poll conducted after the news of Trump’s diagnosis broke, compared to Biden’s 9-point advantage in a poll conducted after the debate but before the president’s diagnosis. The John Zogby Strategies poll that showed Biden ahead by only 2 points nationally was also conducted after the COVID-19 news, although that’s not a pollster I’d give a lot of credence to.
The series of state polls by the New York Times and Siena College are also worth mentioning here, since they’re the only live-caller polls in the mix apart from the NBC/WSJ one — and since they were conducted partly after Trump’s COVID-19 diagnosis. And these contain strong results for Biden, showing him with leads of 8 points in Arizona, 7 points in Pennsylvania and 5 points in Florida– above his polling averages in each state. However, they’re still smaller leads than what previous NYT/Siena polls of those states found — Biden led by 9 points in their Arizona poll in September, for instance.
This raises an interesting question: Should you focus more on how a poll compares to a pollster’s previous poll or how it compares to other polls in that state? There isn’t really a right answer here, it’s a mix of both. But to explain how our model views this, in the case of the September Arizona poll, a 9-point lead had been a particularly strong result for Biden to the point of being a mild outlier. So the most recent NYT/Siena poll counted as a good result for him on balance, as it confirmed he has a clear lead there now, and he gained ground in our Arizona polling average.
Finally, keep in mind that the reason it’s still so hard to understand the effects of the president’s COVID-19 diagnosis is because it is an evolving story. It’s not just his prognosis, which can vary from day to day under the coronavirus, but also the news about the cluster of cases at the White House and the lack of reliable information about his condition. So if the public wasn’t initially sympathetic to Trump — and they mostly weren’t, in part because a large majority of the public thinks Trump didn’t take appropriate precautions — everything we’ve learned since then probably isn’t going to make public sentiment better for him.
With all that said, the situation remains unpredictable. This year, of course, unpredictable news events haven’t always had a large effect on the polls.