Skip to main content
Menu
Why Early Senate And Governor Polls Have Plenty To Tell Us About November

According to a recent survey from Blueprint Polling, Republican Herschel Walker is leading Democratic Sen. Raphael Warnock 49 percent to 45 percent in Georgia’s 2022 Senate race. According to another from Emerson College/The Hill/Nexstar, Texas Gov. Greg Abbott, a Republican, leads Democratic challenger Beto O’Rourke 52 percent to 45 percent. And according to yet another from John Zogby Strategies/EMI Research Solutions/Unite NY, New York Gov. Kathy Hochul, a Democrat, has 14-to-19-point leads over an array of possible Republican opponents.

These are surely good signs for Walker, Abbott and Hochul … or are they?

These polls, like all the others you’ve seen of the 2022 general election so far, were conducted seven months or more before the general election. This means that there is still plenty of time for the trajectory of a race to change — whether due to news events, campaign strategy or shifts in the national environment. But while these polls are still from very early in the election cycle, they’re already pretty useful at this stage — you just have to bear in mind that, like all polls, they come with a margin of error.

That’s according to our analysis of general election polls for Senate and governor conducted in the first six months of every election year going back to 1998. Using over 2,000 polls, we built polling averages for 261 Senate races and 225 gubernatorial contests that had at least one survey in the first six months, finding that these polls have a fairly strong relationship to the final results in November, as the chart below shows.1

It’s important to note, though, that a “fairly strong relationship” is not the same as “every poll completely nailed the final margin.” Senate polls conducted during the first six months of an election year had a weighted average error of 8.2 percentage points — meaning the average polling margin between the Democratic candidate and Republican candidate was 8.2 points off the actual margin. Gubernatorial polls conducted during this span had a similar weighted average error of 8.6 points. That’s not super precise, but it’s also not much worse than polls conducted in the final three weeks of the 1998-2020 campaigns, which had a weighted average error of 5.4 points.

Early polls have also tended to “call” races correctly more often than not this far out, though that’s a bit of a quick-and-dirty way to understand polling accuracy. In a weighted average, early Senate polls have identified the eventual winner 76 percent of the time, while early gubernatorial polls have been right 78 percent of the time. This is only marginally worse than polls conducted in the last three weeks of the campaign, which averaged out to 81 percent for the Senate and 83 percent for governor races from 1998 to 2020.

But there’s a catch with the topline numbers for early surveys: Polls that show close races for Senate or governor — those with margins between 0 and 5 points — are essentially coin flips when it comes to being correct in November, as the table shows. The likelihood that polls nailed the eventual winner in races where the candidate has a 5-to-10 point edge or a double-digit lead is much higher, though, at roughly 75 percent and 90 percent of the time, respectively.

Early surveys often miss the mark in close races

Weighted average share of polls that correctly identified the winner in the first six months of the election year, by how close the poll showed the race to be, 1998-2021

Share of correct calls in races for …
Poll margin Gov. Senate
0 to 5 points 54% 51%
5 to 10 points 76 73
10+ points 91 92

Pollsters get half-credit if they showed a tie for the lead and one of the co-leading candidates won.

Excludes polls sponsored by organizations that operated on behalf of a candidate, party, campaign committee, PAC, super PAC, hybrid PAC, 501(c)(4), 501(c)(5) or 501(c)(6) organization that conducted a large majority of its political activity on behalf of one political party. Also excludes polls of races that featured jungle primaries or where the Democratic and Republican candidates didn’t finish first and second. For relevant races that went to runoffs, we compared non-runoff polls to the general election outcome before the runoff. To avoid giving a more prolific pollster too much influence, polls were weighted by 1 divided by the square root of the number of polls each pollster conducted.

Source: Polls

One more point in early polls’ favor: We found that they did not consistently overestimate either party — i.e., their weighted average bias was not disproportionately skewed toward Democrats or Republicans. Overall, early Senate polls conducted in the first six months of all races from 1998 to 2021 had only a 0.1-point weighted average bias toward Republicans, while early gubernatorial surveys had just a 0.02-point lean toward the GOP.

This doesn’t mean that some individual cycles didn’t have significant biases — only that early polls in those cycles weren’t any more biased than polls later in those cycles. (If you’re a regular FiveThirtyEight reader, you’re probably already familiar with the fact that the late polls overestimated Democrats in both 2016 and 2020; well, it turns out the early polls did, too.) 

In the two tables below, we lay out the weighted average bias, weighted average error and weighted average correct-call percentage for every election cycle since 1998:

Most govs. are elected in midterms, when bias has varied a lot

Weighted average error, bias and share of correct calls in polls of gubernatorial races conducted in the first six months of every election year, since 1998

Cycle No. of polls Avg. Error Avg. Bias Correct Calls
1998 38 12.8 R+11.2 85%
2000 8 5.7 D+4.3 88
2002 94 9.9 D+0.5 79
2004 14 8.2 D+2.0 63
2006 193 10.2 R+5.2 82
2008 43 5.6 D+1.2 88
2010 213 7.0 R+0.3 80
2012 30 8.0 R+2.1 60
2014 160 9.1 D+6.9 73
2016 24 4.5 D+2.5 77
2018 59 6.0 R+1.4 84
2020 27 8.9 D+7.4 100
1998-2021 994 8.6 R+0.02 78

Pollsters get half-credit if they showed a tie for the lead and one of the co-leading candidates won.

Excludes polls sponsored by organizations that operate on behalf of a candidate, party, campaign committee, PAC, super PAC, hybrid PAC, 501(c)(4), 501(c)(5) or 501(c)(6) organization that conducts a large majority of its political activity on behalf of one political party. Also excludes polls of races that featured jungle primaries or where the Democratic and Republican candidates didn’t finish first and second. For relevant races that went to runoffs, we compared non-runoff polls to the general election outcome before the runoff. To avoid giving a more prolific pollster too much influence, polls were weighted by one divided by the square root of the number of polls each pollster conducted.

Relevant off-year gubernatorial elections from 2003 to 2021 were included in the overall analysis, but those cycles are excluded from the table due to small sample sizes. Data from 1999 and 2001 was unavailable. A total of 91 early polls were conducted in those 10 odd-year cycles.

Source: Polls

Most govs. are elected in midterms, when bias has varied a lot

Weighted average error, bias and share of correct calls in polls of gubernatorial races conducted in the first six months of every election year, since 1998

Cycle No. of polls Avg. Error Avg. Bias Correct Calls
1998 38 12.8 R+11.2 85%
2000 8 5.7 D+4.3 88
2002 94 9.9 D+0.5 79
2004 14 8.2 D+2.0 63
2006 193 10.2 R+5.2 82
2008 43 5.6 D+1.2 88
2010 213 7.0 R+0.3 80
2012 30 8.0 R+2.1 60
2014 160 9.1 D+6.9 73
2016 24 4.5 D+2.5 77
2018 59 6.0 R+1.4 84
2020 27 8.9 D+7.4 100
1998-2021 994 8.6 R+0.02 78

Pollsters get half-credit if they showed a tie for the lead and one of the co-leading candidates won.

Excludes polls sponsored by organizations that operate on behalf of a candidate, party, campaign committee, PAC, super PAC, hybrid PAC, 501(c)(4), 501(c)(5) or 501(c)(6) organization that conducts a large majority of its political activity on behalf of one political party. Also excludes polls of races that featured jungle primaries or where the Democratic and Republican candidates didn’t finish first and second. For relevant races that went to runoffs, we compared non-runoff polls to the general election outcome before the runoff. To avoid giving a more prolific pollster too much influence, polls were weighted by one divided by the square root of the number of polls each pollster conducted.

Relevant off-year gubernatorial elections from 2003 to 2021 were included in the overall analysis, but those cycles are excluded from the table due to small sample sizes. Data from 1999 and 2001 was unavailable. A total of 91 early polls were conducted in those 10 odd-year cycles.

Source: Polls

But good luck guessing the direction of the bias of early 2022 polls. True, in the last four cycles, early Senate polls have been about 3 to 6 points too good for Democrats. But in three of the four cycles before that, early Senate polls had overestimated Republicans — and early gubernatorial polls overestimated Republicans in 2018, too. Moreover, as the 2018 cycle showed, one set of polls could be skewed toward one party, while surveys for another office could end up biased toward the other party. Though the 2022 electoral environment is shaping up to be advantageous for the GOP, just how accurately the polls gauge races remains to be seen.

Looking ahead to the 2022 election, though, there are still some takeaways for us based on our analysis of early polls in previous cycles. For instance, there’s not much we can say about Georgia’s Senate race, as Walker leads Warnock by about 2 points, on average; we found in previous cycles that when a candidate led by 0 to 5 points, the accuracy of polls was essentially a coin flip. We likely have a better sense of the ultimate outcome in Texas’s governor’s race, though, as Abbott currently leads O’Rourke by about 6 points, on average, which has historically given candidates a 3-in-4 shot at winning. Finally, in New York’s gubernatorial contest, we don’t know for sure whom each party will nominate, but assuming the apparent frontrunners do meet in November — Hochul and GOP Rep. Lee Zeldin — the early polls definitely point toward a Hochul victory, as she leads Zeldin by about 13 points, which is the sort of advantage that historically has signaled a candidate has a 9-in-10 chance of victory.

So can you trust the polls this far out? Sure you can, but be careful about how you interpret them. Polls are not a precise measurement of where a race will end up, even in the final days of an election. That said, they’re pretty telling, even many months before the general election.

Footnotes

  1. The correlation between the Senate polling averages and results was .897, while the correlation between the gubernatorial polling averages and results was .839.

Geoffrey Skelley is an elections analyst at FiveThirtyEight.

Nathaniel Rakich is a senior elections analyst at FiveThirtyEight.

Comments

Latest Interactives