We’ve discovered that people are a little bit obsessed with the presidential race in Utah. An article that my colleague Benjamin Morris wrote last week about Evan McMullin, an independent candidate who is on the ballot there and competitive in recent polls, unexpectedly turned out to be one of the most popular features we’ve written this year at FiveThirtyEight. In this article, I’ll provide a more technical explanation of how our model is forecasting McMullin and why he has a relatively challenging path — and also, one important way in which our forecast might be underrating his chances.
This spring, I spent a lot of time analyzing third-party candidates and how their polls behave in presidential and U.S. Senate races. They’re tricky to forecast for a few reasons:
- Polls tend to overstate the performance of third-party candidates early in the race — but not necessarily late in the race.
- Third-party candidates have asymmetric probability distributions — see, for example, the chart below for how our model forecasted Gary Johnson’s vote in New Mexico earlier this year. In our average simulation, Johnson’s projected vote share was about 10 percent of the vote. But that average was made up of a majority of cases in which he faded and received less than 10 percent, plus a handful where he broke out and won 25, 30, 35 percent of the vote or more.
- A third-party candidate’s polls can behave differently at different thresholds of the vote. A candidate polling in the low single digits — say, 2 or 3 percent — is often just a placeholder for “undecided” and their support can fade to close to zero by Election Day, especially in a competitive race. When a candidate is in the mid- to high single digits, conversely, or certainly in the double digits, their standing in polls is more likely to reflect genuine support.
- A more important threshold comes somewhere in the range of 25 percent of the vote. Third-party candidates above this range can plausibly win in a three- or four-way race, while candidates below it can only spoil the victory for someone else or serve as protest votes. That’s why you usually don’t see a lot of races with results like: the Republican 39 percent, the Democrat 37 percent, and the independent 24 percent. In a case like that, the independent wasn’t quite close enough to have a shot to win the race herself, but she also had enough support that her voters could easily have tipped the balance between the Democrat and the Republican.
You’ll notice that McMullin, who has received between 20 and 29 percent of the vote in Utah in four recent polls, is close to the threshold I described above. That puts him on something of a precipice: He’s likely to either gain support or lose it instead of staying where he is right now.
What makes this tricky — and why the model probably underestimates McMullin’s chances — is that he’s a late arrival on the scene, having only qualified for the Utah ballot in August and having only started to attract significant attention recently. Typically in mid-October, if you saw an independent polling at 20-something percent, you’d assume he was on his way down after once having been more competitive. McMullin, however, has the potential to benefit from a feedback loop as more people hear about his candidacy and consider him to be a viable option. And it’s interesting that the most recent poll of the bunch, from Rasmussen Reports, gave McMullin his best number. Our model will respond aggressively if further polls find McMullin in the high 20s or low 30s instead of at 20 percent of the vote. You can already see some of the difference in that our now-cast, which weights recent polls more heavily and makes more sympathetic assumptions about third-party candidates, gives McMullin a considerably better chance than the polls-only and polls-plus models do.
McMullin presents some other modeling challenges. Unlike Johnson, who’s drawing support relatively evenly from the two major candidates (perhaps slightly more from Hillary Clinton, although that’s varied over time), McMullin is explicitly appealing to voters who would ordinarily vote Republican for president but who don’t like Donald Trump. Also unlike Johnson, who’s on the ballot everywhere, McMullin is only on the ballot in 11 states and is probably only a prospect to win in Utah, with its heavily Mormon population. (McMullin is Mormon, whereas Mormon voters have a lot of problems with Trump.) Idaho is another possibility, although its Mormon population is considerably smaller than Utah’s and there have been no recent polls there with McMullin on the ballot.
So here’s how we’re handling McMullin in our forecast:
- We’re only forecasting McMullin in Utah for now. We’ll add additional states such as Idaho if and when McMullin becomes a factor in the polls there.
- In Utah, we’re only using polls that include McMullin — all other polls are weighted to zero.
- McMullin isn’t eligible for some of the fancier adjustments our model applies, such as the house effects adjustment, because there isn’t enough data to do anything all that complicated with him. However, the model puts more of a premium on recent polls when estimating McMullin’s vote, as a substitute for our regular trend line adjustment.
- Since third-party candidates are a significant contributor to polling error and polling volatility, the uncertainty in Utah is higher in the model because of McMullin’s presence.
- The model uses regression analysis — comparing how the other candidates’ vote shares vary with McMullin’s vote — to infer where McMullin is taking his votes from. Currently, it shows about 50 percent is coming from Trump, although with meaningful shares also from Clinton, Johnson and undecided.
- The model uses these correlations in its simulations. For instance, in simulations where McMullin does well, more of his gains come from Trump than from Clinton.
- As Morris mentioned, McMullin is eligible to be chosen president by the House of Representatives if he finishes in the top three in electoral votes (meaning that he wins Utah or some other state) and no candidate gets a majority of 270 electoral votes. Where these cases come up, the model gives McMullin the presidency 10 percent of the time. (The 10 percent figure is totally arbitrary, but it would be an unprecedented situation so I don’t know what better assumption to make.) While the polls-only model currently has McMullin winning Utah about 6 or 7 percent of the time, the overall parlay is rather unlikely and results in McMullin winning the presidency in only about 1 in every 5,000 simulations.
Why isn’t McMullin’s probability higher? Well, for the time being he’s behind, at least based on the polling average. Polling geeks have focused on the Rasmussen Reports and Y2 Analytics poll that showed a close three-way race in Utah, but less on the YouGov and Monmouth polls that still had Trump ahead (way ahead in the case of YouGov’s poll). A simple average of the four recent polls yields a result of Trump 32 percent, Clinton 26 percent and McMullin 23 percent, putting McMullin within striking distance but also in third place with a 9-point deficit to make up in three weeks.
Another complication is that in cases where Trump is doing badly enough to lose Utah, he’s probably getting crushed by Clinton overall. Yes, the Mormon vote is especially important in Utah and is something of a unique factor. But there are also enough Mormons in Colorado, Nevada and Arizona to potentially swing the outcomes in those states. If Clinton’s winning Arizona — well, it’s probably a full-blown landslide and the election isn’t going to the House.
One could even argue that the whole Utah obsession is misplaced, given that various traditionally red states from Arizona to Alaska are more likely than Utah to end up in Clinton’s column, according to our model. Still, the mere fact that pollsters are thinking about Utah as a competitive state is remarkable. And I wouldn’t be surprised to see McMullin gain further ground in the next round of polling.
This is one of the reasons what we don’t explicitly forecast Jill Stein’s vote, although the model assigns some of the vote to “other” in states where candidates beyond Trump, Clinton and Johnson are on the ballot.
This is one of the reasons what we don’t explicitly forecast Jill Stein’s vote, although the model assigns some of the vote to “other” in states where candidates beyond Trump, Clinton and Johnson are on the ballot.
There’s really no excuse not to include McMullin in a Utah poll, but some of the 50-state polls that list the same set of candidates in every state don’t.
This is one of the reasons what we don’t explicitly forecast Jill Stein’s vote, although the model assigns some of the vote to “other” in states where candidates beyond Trump, Clinton and Johnson are on the ballot.
There’s really no excuse not to include McMullin in a Utah poll, but some of the 50-state polls that list the same set of candidates in every state don’t.
As a heads-up, we may add back in some of our other adjustments if we begin to get a lot more data on McMullin and how various pollsters are handling him.