Election Update: It’s Cherry-Picking Season

As of our most recent Election Update this weekend, I noted that polls since last week’s debate had been — this is a scientific term — “pretty weird.”

Well, it’s time to revise that statement. There have been a whole bunch of new polls over the past couple of days. And they’ve been not just pretty weird … but really weird. As far as I’m concerned, that means it’s exactly the sort of time when it’s helpful to take an average instead of fixating on individual polls.

On balance, national polls have been pretty good for Bernie Sanders — but there’s a lot of variation from survey to survey. On Wednesday morning, for instance, a CNN poll came out showing him leading Joe Biden nationally and having gained 7 percentage points from the previous CNN poll in December. But a couple of hours later, YouGov’s weekly tracking poll came out showing Sanders in third place — or having lost ground since last week’s debate.

If you’ve come looking for confident assertions about which poll is “right” and which one is “wrong” — well, that’s just not how we do things around here. And any such claims would be a bit ridiculous given the inherently high margin of error on primary polls. Primary and caucus polling is always a struggle, so the hope is that by accounting for a wider range of pollsters and methods, an average will prove to be a bit less wrong than any individual poll might be. At a bare minimum, averaging or aggregating polls increases the sample size, which is a relevant factor since primary polls often use considerably smaller sample sizes than general election ones.

What you don’t want to do, of course, is to cherry-pick data. There’s actually been quite a bit of post-debate polling, and if Sanders had actually gained 7 points nationally, as the CNN poll shows, we would have seen more signs of it by now.

It would be an equally big mistake, however, to “throw out” or ignore the CNN poll. CNN has not shown especially strong results for Sanders before, and CNN’s pollster, SSRS, is reasonably highly rated. It’s a sign that a pollster is doing honest work when it’s willing to publish a poll that differs a bit from the consensus. So stick the CNN poll in the average instead … and you’ll see there’s still some good signs for Sanders.

National polls show Sanders and Bloomberg gaining, others flat

In fact, at 20.4 percent, Sanders is in his strongest position in our national polling average since April. State polls have been much more of a mixed bag, with Sanders having fallen slightly in our Iowa and New Hampshire polling averages since the debate — but we’ll cover those in the next section. A bit more about those national polls first.

In addition, to CNN and YouGov, Morning Consult and Monmouth University also released national polls on Wednesday morning. These join earlier post-debate national polls from SurveyUSA and Ipsos. Here’s a table showing those polls for the top six candidates:

And here’s a companion table showing the change from the previous pre-debate poll for each pollster. Note that the pre-debate poll wasn’t necessarily that recent; the last time that SurveyUSA had polled the race nationally was in November, for instance.

In the tables, you can see how a simple average of the six most recent polls compares to the much fancier FiveThirtyEight polling average: They’re really pretty darn similar. There are some differences, though. For instance, it’s worth noting that the SurveyUSA poll had about three times (1086) as many respondents as Monmouth (372) and more than twice as many as CNN (500); our averages account for that by giving SurveyUSA more weight. House effects are also something of a factor; for instance, SurveyUSA tends to show good numbers for Biden, Morning Consult tends to show good numbers for Sanders, YouGov tends to show good numbers for Elizabeth Warren; and Ipsos tends to show poor numbers for all candidates except Sanders. Those explain some of the differences between the polls, too.¹

But none of this really matters that much. As you can see, the FiveThirtyEight average and a simple average of the six post-debate polls produce highly similar results. Likewise, the average change in the polls is pretty similar regardless of which method you use. Take a simple average of the change in Sanders’s numbers since the last time these six pollsters surveyed the field, and he’s gained 2.0 percentage points since the debate. Similarly, he’s gained 1.7 percentage points in the fancy version of the FiveThirtyEight average, calculated since Jan. 14, the day of the debate. Michael Bloomberg has also gained ground regardless of what method you choose and is close to catching Pete Buttigieg in our national average.

State polls haven’t been great for Sanders, though

There have also been quite a few state polls since the debate, however, and they don’t tell a terribly consistent story with the national polls. They’re actually on the weak side for Sanders. Iowa polls from Neighborhood Research and Media and David Binder Research, both conducted since the debate, have Sanders in 5th and 4th place, respectively in the Hawkeye state. But Sanders does have some decent excuses here; the David Binder poll has generally been one of his worst ones in Iowa, and the Neighborhood Research poll has a small sample size and is from a Republican pollster that hasn’t previously released data in Iowa (as such, it receives a relatively small weight in our model).

Nonetheless, based on recent Iowa polls — both before and after the debate — Iowa would appear to be Biden’s state to lose more than Sanders’s. Biden has been ahead or tied for the lead in 4 of the 5 Iowa polls since the new year, as compared with just two for Sanders. (Although one of the polls that had Sanders ahead was the highly-rated Selzer & Co. poll.) In fact, Sanders has basically slipped into a three-way tie for second place in Iowa with Warren and Buttigieg.

In New Hampshire, true to the theme, the polling has also been weird. Two new polls conducted wholly or partially since the debate, from Emerson College and Suffolk University, each show Sanders leading there — but having lost ground relative to the previous editions of the same polls. Instead, these polls show growth for candidates outside of the top three, such as Buttigieg and Amy Klobuchar. And in the case of the Suffolk poll, there are also a large number of undecided voters. (Sanders, despite being the leading candidate in the Suffolk poll, has only 16 percent of the vote in the poll.) Sanders leads in our New Hampshire polling average, but no candidate should feel particularly secure in the Granite State.

Wednesday also saw the release of polling in four delegate-rich Midwestern swing states from a consortium of universities there, Baldwin Wallace University, Oakland University and Ohio Northern University. Sanders led in Wisconsin, but Biden held leads in Ohio, Michigan and Pennsylvania.

How much do national polls matter?

Overall, our forecast just hasn’t changed that much since we first released it two weeks ago. Biden has a 43 percent chance of a delegate majority, followed by Sanders at 20 percent, Warren at 14 percent and Buttigieg at 8 percent. The chance of no candidate winning a majority of delegates is 15 percent. Technically, Biden is up very slightly and Sanders is down very slightly since we first launched the forecast, but the differences are small and not worth spending a ton of time worrying about.

If you look carefully, though, you can see how the model tends to privilege state polls over national polls (or at least it does so right now, with Iowa set to vote in less than two weeks). For example, Sanders did gain ground in the model following the release of the Selzer poll of Iowa on Jan. 10, but — despite his standing in national polls improving — he’s given those gains back because of an underwhelming series of polls in Iowa and New Hampshire.

You might be wondering: How does the model even use national polls? There is no national primary, after all. But national polls have several uses in the model. To simplify, here are the three most important ones:

1. 1. We use national polls to calculate a trendline adjustment, which can influence states that haven’t been polled recently. For instance, if there are no recent polls of Oregon, but Sanders has gained 3 points in national polls since the last time Oregon was polled, the model will assume he’s gained ground in Oregon, too.
   2. National polls are used as a baseline to calibrate the bounce a candidate will potentially receive after winning or finishing strongly in a state. The twist is that the higher a candidate’s standing in national polls, the less her bounce after winning a state (and the more she might decline after losing a state). Empirically, the bounce that a candidate gets after winning a state depends strongly on expectations, and national polls are a good proxy for voter and media expectations. Less abstractly, the fact that Sanders is now seen as a national front-runner means that — as is also the case with Biden in Iowa — he might gain less ground following a win there, and lose more ground nationally if he finishes in anything other than first place or a strong second. Warren or Buttigieg, on the other hand, would probably be seen more as underdogs, and those sort of candidates have historically gotten bigger bounces after winning Iowa.
   3. In various implicit and explicit ways, national polls help the model to predict the outcome of states where there isn’t much polling.

So basically, factors No. 1 and No. 3 tend to help a candidate in the model when their national polls are strong, but factor No. 2 (rising expectations) actually hurts them. To be clear, our experience with the model so far — keep in mind that this is the first time we’ve run a full-fledged primary forecast — is that No. 1 and No. 3 usually outweigh No. 2. In other words, a candidate would usually prefer to gain ground in national polls, as far as the model is concerned. Still, there is some ambiguity about the influence of national polls in the model, especially when there is a lot of recent state polling and so the timeline adjustment (No. 1) doesn’t have as much impact.

Essentially, this leaves us with three plausible interpretations of the post-debate polling:

• National polls tell the true story of the race, and the slightly quirky set of recent Iowa and New Hampshire polls since the debate are misleading. If so, Sanders should expect to do better in the next set of Iowa and New Hampshire polls. This would be good news for Sanders.
• All of this is noise, and none of the candidates’ positions have changed much since the debate. This would be neutral news for Sanders.
• Whether or not Sanders is gaining in national polls, Iowa and New Hampshire have their own dynamics, and Sanders does not appear to be closing strongly there. This would be bad news for Sanders.

Polling since the debate just doesn’t provide a lot of clarity on which one of these stories is correct. So we’ll have to wait and see if the picture clears up by the weekend.

Is our primary system democratic?