This is the closest thing I’ve taken to a day off in quite some time, but briefly:
As I hinted yesterday, the amount of variance in each simulation run now differs from state to state. There are actually two different components to this. The first is how responsive a state is to national trends. We had already figured out a way how to estimate this. However, we now apply it specifically to each simulation. For example, let’s say that New Hampshire polls move 120 percent as much as national polls. If in simulation #3,268, Obama’s national trend has moved downward by 5 points, New Hampshire’s polls will move 120 percent that much, or down 6 points. If in simulation #7,008, Obama’s national trend has moved upward by 10 points, New Hampshire’s polls will move up by12 points. It’s as simple (or as complicated) as that.
Separately, however, there is also a question of how much variation there is within a given state’s polling, period. You could have a state where the polls are relatively uncorrelated with national trends, but where the polls nevertheless seem to fluctuate wildly, marching somewhat to their own drummer. The way we account for this type of variance is to take the standard deviation across all polls conducted in a state after having stripped out the national trendline. Then, we run our demographic regression against these standard deviations to see whether anything systematic seems to be driving the amount of volatility in a given state’s polling. It turns out that there are a few such things: variance tends to be lower in states with large numbers of African-Americans, for instance, but higher in states with large numbers of elderly voters.
The most important implication of this is that the polls are liable to be more stubborn in the Deep South (excluding Florida) than they are elsewhere in the country. So even though Obama has whittled down McCain’s lead to the single digits in states like Georgia, Mississippi, and South Carolina, those are going to be tough points for him to make up. On the other hand, the polls have been quite volatile in Appalachia, where you have a lot of conflicted, downscale voters who are not particularly fond of either of these candidates. So, even though we show McCain with the same 8-point lead in both Georgia and West Virginia, our model gives him a 91 percent chance of hanging on to win Georgia, but just a 75 percent chance of winning West Virginia.
You will notice, by the way, that this second adjustment doesn’t distinguish between cases where the polls vary a lot over time, and cases where they vary a lot at the *same* time (as they do in Florida right now, for instance). That’s perfectly okay, because both things increase our degree of uncertainty about exactly what’s going to happen in November.
Lastly, I have swapped out a couple of variables in the 538 regression analysis. The ‘partisan’ variable has been replaced by a liberal-conservative Likert scale for each state drawn from 2004 exit polling. This seems to provide slightly more unique information to the model than the partisan ID index, particularly as partisan identification tends to change more quickly over time than one’s political philosophy. I have also added a variable for Hillary Clinton’s performance in the primaries (the results from caucus states are adjusted). Yes, all else being equal, Obama does worse where Hillary had done better.
p.s. Happy 232nd birthday, America!