The United States of America is a constitutional republic made up of 50 states and the District of Columbia.
I’m sure you knew that. But it seems as though pollsters might have slept through social studies class. In theory, state polls and national polls must ultimately agree with each other: the whole must equal the sum of the parts. In practice, they leave a different impression about the state of the presidential race.
Consider that as of Tuesday afternoon, President Obama’s lead in the RealClearPolitics average of national polls was 1.3 percentage points over Mitt Romney.
But Mr. Obama led by a mean of 3.5 points in the RealClearPolitics averages for the 10 states (Ohio, Virginia, Florida, Pennsylvania, Colorado, Iowa, Nevada, Michigan, New Hampshire and Wisconsin) that are most likely to determine the election outcome, according to our “tipping point index.” If the list is expanded to cover the five other marginally competitive states where RealClearPolitics calculates a polling average — Arizona, Minnesota, Missouri, New Mexico and North Carolina — Mr. Obama’s lead averages 3.1 points, according to the numbers.
While that isn’t an enormous difference in an absolute sense, it is a consequential one. A one- or two-point lead for Mr. Obama, as in the national polls, would make him barely better than a tossup to win re-election. A three- or four-point lead, as in the state polls, is obviously no guarantee given the troubles in the economy, but it is a more substantive advantage.
The difference isn’t an artifact of RealClearPolitics’s methodology. The FiveThirtyEight method, which applies a more complicated technique for estimating polling averages, sees broadly the same split between state and national polls.
Or if you prefer a bare-bones approach, you can take a count of the polls conducted in swing states: how many show leads for Mr. Obama, and how many put Mr. Romney on top?
In polls from the top 10 tipping point states since June 1, Mr. Obama has led by any margin in 43 surveys, while Mr. Romney has held the lead in nine of them.
Since July 1, the discrepancy has been even clearer: Mr. Obama has held leads in 19 polls from these states, and Mr. Romney two.
Often, Mr. Obama’s leads in these polls have been small, but that still doesn’t seem consistent with the notion — as the national polls seem to express — that the race is about tied. We need some way to explain the results, or at least a strategy to deal with the uncertainty this introduces.
Perhaps the most common assumption is that if Mr. Obama is performing comparatively well in swing state polls, he might have some advantage in the Electoral College in relation to the national popular vote. Perhaps we are on track for a split outcome, as we had in 2000, but this time with Democrats benefiting from the idiosyncrasies of the Electoral College.
It’s certainly possible that we’ll see such an outcome, but historically speaking, it is not something you would want to bet on. Besides 2000, such a split occurred just two other times — in 1876 and in 1888 — since the popular vote came to be widely reported. That’s just 6 or 7 percent of the time. The FiveThirtyEight model thinks the odds of such a split are not much better than that this year.
Instead, there are two other hypotheses that might also explain the difference, in whole or in part. They are banal and even boring, so simple that they might be overlooked.
One of these hypotheses is that the state polls are overrating Mr. Obama somehow. The other is that the national polls are underrating him. Or it could be that both of these ideas are right to a degree, and the truth lies somewhere in the middle.
In this column, I will describe a method for comparing state and national polls: they are more suitable to a direct comparison than you might think. On Wednesday, we’ll return to the question of how to derive the best prediction of the election outcome from the two sets of numbers.
***
The reason that state polls and national polls are fairly easy to compare is that the relative order of the states is extremely consistent from year to year. We know more or less how Vermont, Iowa and Wyoming, along with the other 47 states, will line up relative to the national trend.
Let’s define a simple quantity called the relative popular vote. It measures the popular vote in a given state relative to the national popular vote.
For instance, in 2008, Mr. Obama won Ohio by 4.6 percentage points. But he won the national popular vote by a slightly wider margin, 7.3 points. Subtracting 7.3 points from 4.6 yields negative 2.7. That means that Ohio was slightly Republican-leaning — by 2.7 points — relative to the national trend.
Knowing what the relative popular vote had been in each state in 2004 would have sufficed to explain about 90 percent of its relative popular vote in 2008, even without consulting any polls. This was no anomaly; these numbers are very consistent from election to election. We could show you the same chart for pretty much any of the last six or seven elections, and it would look about the same.
It is possible — and, in fact, desirable — to take advantage of this property when building a forecast model.
For instance, my analysis has found that when polls in a state are out of line with its relative popular vote from the past election or two, it often snaps back into place by November, unless this pattern is explained by another factor like the state’s being the home of one of the candidates. This is why our forecast model gives quite a bit of weight to what I call “state fundamentals,” the largest of which is simply how the state has voted recently.
You might call this the “top-down” method. Figure out the national trend, then back into an estimate of where the vote should be in each state based upon that.
Say, for example, that Washington state typically votes about 10 points more Democratic than the country as a whole. If Mr. Obama leads by two points nationally, that would imply that he would win in Washington by about 12 points.
But can’t this also work the other way around? Say you have a poll showing Mr. Obama ahead by 12 points in Washington, and you know that Washington is 10 points more Democratic than the national average.
Can’t you infer from that poll that Mr. Obama is ahead by about two points nationally?
This “bottom-up” approach is also informative, it turns out. Or to put it another way, a poll in one state tells us something about the conditions of the
race in the other 49 states.
Of course, this method needs to be applied quite carefully. Inferring the national trend from the polls in just one state will introduce a lot of noise into the system.
But if you look at the state polls in all 50 states — or at least all states where there is a significant amount of polling — and do so in a systematic way, it can tell you quite a bit about where the national race stands. In fact, as we’ll discuss in the follow-up column, the “bottom-up” method will sometimes give you a better estimate of the national popular vote than you’ll get from the national polls themselves.
Let me give you a more detailed sense for how this works. First, we’ll apply a relatively simple version of this calculation, then I’ll give you the exact method that the FiveThirtyEight model uses.
For the simple method, we’ll look at the 25 states that have been polled at least twice since May 1. Since then, the national race has been fairly stable, at least in our estimation.
In each of these states, I’m listing what I call the FiveThirtyEight modified polling average. The modified polling average starts out with our regular polling average, which weights the polls based on how recently they were conducted, the reliability of the pollster and the sample size of the poll. Then it makes three other adjustments: a likely voter adjustment (which reflects how polls of registered rather than likely voters typically underestimate the standing of the Republican candidate), a trendline adjustment (which reflects changes in the overall standing between the two candidates since the poll was conducted) and a “house effects” adjustment (which corrects for systematic biases toward either the Democratic or Republican candidates among certain pollsters). We’ve explained how each of these adjustments function in the past so I won’t dwell on them now.
We can then compare the modified polling average with the relative popular vote of the state in 2008.
In New York, for example, the modified polling average put Mr. Obama 23.3 points ahead as of Tuesday’s forecast. In 2008, New York was 19.6 points more Democratic than the country as a whole.
By subtracting 19.6 points from 23.3 points, we get what I call an implied national popular vote. In this case, the result is Obama plus 3.7 points. In other words, the New York polls are most consistent with a race in which Mr. Obama leads by 3.7 points nationally.
This calculation varies some from state to state. The implied national popular vote derived from the polls in Arizona is Obama plus 8.7 points. (In this case, the high figure results in part from the fact that Arizona was John McCain’s home state, meaning that it is easier for Mr. Obama to make up ground relative to the 2008 finish.) Whereas in Oregon — which Mr. Obama won easily in 2008 but where polls have shown a relatively tight race this year — the implied national popular vote is Romney plus 4.2 points.
These idiosyncratic cases tend to even out if we perform this calculation across all the states. Furthermore, we can weight the states based on their turnout in 2008: a deviation in the vote in Rhode Island or Wyoming will matter much less than one in California, Texas or Florida in terms of the national popular vote.
If we perform this calculation across all the states with a significant amount of recent polling, we come to a result of Obama plus 2.9 points. In other words, the totality of state polling — dozens and dozens of state polls, looked at in a careful way — is most consistent with a race in which Mr. Obama leads Mr. Romney by about three points nationally, according to this approach.
The method that our forecast model uses is similar to this, but it introduces some more nuances into the calculation.
First, instead of just looking at a state’s relative popular vote in 2008, I calculate something called the FiveThirtyEight Presidential Voting Index. (The name is a takeoff on Cook Political’s partisan voting index, which makes a similar calculation.)
The FiveThirtyEight Presidential Voting Index differs from the simple method in a few important ways:
Although this calculation might seem involved, the way to read the voting index is relatively straightforward. In Missouri, for example, the index is Republican plus 6.7 points. What this means is that if the popular vote were exactly tied nationally, we’d expect the Republican candidate to carry Missouri by 6.7 percentage points.
We can use this index to calculate an implied national popular vote, just as we did with the simpler method before.
There are two more wrinkles that the FiveThirtyEight method applies. (Neither is that important, but I’m trying to document the method as completely as possible.) First, it weights each state not just based on the turnout there, but also on the quality and quantity of polling. So — although technically it determines an implied nation
al popular vote for South Carolina — the South Carolina value receives very little weight in the overall calculation since the only poll there was conducted months ago. Second, it adjusts the difference between a state’s modified polling average and its voting index by its elasticity score, which reflects the fact that some states are “swingier” than others, making it easier to make gains at the margin there.
Based on the official version of the calculation, the implied national popular vote from the state polls is Obama by 3.6 points.
In plain English, if you had to guess what the national popular vote was from the state polls alone — without ever looking at a national poll — the best estimate would be that Mr. Obama was ahead in the race by three or four points, even after adjusting for likely voters and for other factors.
This is slightly larger than the lead suggested by national polls alone, which put Mr. Obama ahead by about one or two points instead, according to our method.
In other words, there is some divergence between the “top-down” and “bottom-up” approaches — the totality of state polls implying a somewhat more favorable result for Mr. Obama than the national polls do. Our “fancy” way of looking at the numbers doesn’t resolve the dilemma that I introduced earlier on.
In Wednesday’s column, we’ll explore different hypotheses that might explain these differences, and consider the best policy when they arise.
This post has been revised to reflect the following correction:
Correction: July 25, 2012
Two charts accompanying an earlier version of this post showed President Obama ahead in the polling average of Montana. He is behind in the state, not ahead.