Our latest Senate simulation has the chamber convening in 2011 with an average of 53.4 Democrats (counting Joe Lieberman and Bernie Sanders), 46.1 Republicans, and 0.5 Charlie Crists. This is an improvement for Republicans from our last forecast three weeks ago, which had 55.2 Democrats, 44.2 Republicans, and 0.6 Crists. The changes, however, predominantly reflect several methodological improvements we have made rather than any particular national momentum, although the dynamics of some individual contests are certainly evolving.
The model gives Republicans a 17 percent chance of taking over the Senate if Charlie Crist caucuses with them, up significantly from 6 percent three weeks ago. If Crist does not caucus with them, their chances of a takeover are 12 percent. However, the model does not account for the contingency that someone like Joe Lieberman or Ben Nelson could decide to switch parties, which makes their chances slightly better than we suggest here.
Democrats’ chances of gaining a net of one or more seat and re-claiming a 60-seat majority are 7 percent, down from 12 percent three weeks ago. If they could persuade Charlie Crist to caucus with them, however, their chances would improve to 10 percent.
Methodological improvements. There are four changes to the model from June’s version, which are briefly described below. The first change — the likely voter adjustment — is by far the most important.
Likely voter adjustment. We now notate whether each poll is of likely voters, registered voters, or all adults, and include variables for this in the regression analysis we use to calculate pollster house effects. The regression shows that, holding house effects constant, Democrats do a net of 4 points better in polls of registered voters (with a 95 percent confidence interval of about 2-6 points) than in polls of likely voters, and roughly 7 points better in polls of all adults. So if we took, for example, the recent Ipsos poll of California, a poll of registered voters which showed Barbara Boxer with a 4-point lead over her Republican challenger, Carly Fiornia, we would expect it to show about a tied race if a likely voter screen had been applied instead.
The adjustment works in exactly this fashion: it adds a net of 4 points to the Republican candidate’s margin each time that it encounters a poll of registered voters, and 7 points every time that it encounters a poll of adults (which is very rare in state-level polling.) Polls of likely voters are left unchanged as having a likely voter screen is assumed to be the default condition.
In 2008, I had been reluctant to be overly aggressive about using likely voter polls this far out from the election. In fact, when the pollster had presented us with a choice of a registered voter and a likely voter poll, we went with the registered voter numbers until Labor Day. The reason for my change of heart is twofold. First, by the time we get to November, essentially all polls will be of likely voters. In other words, we are essentially predicting what the polls are likely to show in due time anyway, once all pollsters have switched to a likely voter model. This should lend additional stability to the forecasts, as they will converge toward their final estimates more quickly. Secondly, there is substantial quantitative and qualitative evidence of an enthusiasm gap between Democratic and Republican voters, which we would quite naturally expect to be reflected in likely voter polls but not so in polls of registered voters. (It is probably also the case that registered voter polls generally somewhat underestimate Republicans’ performance in midterm elections even when the mood of the country is fairly neutral, but that is especially likely to be the case in a year like this one, where enthusiasm is imbalanced).
Because of the way our model is calibrated, this adjustment does make a fair amount of difference: it is most of the reason that Republicans have a somewhat better forecast now than they did three weeks ago. But note that the adjustment is self-correcting; if, as other pollsters switch over to likely voter models, they do not show an enthusiasm gap, its effect will be reduced in our model.
We are tracking to see whether a similar adjustment might be warranted for polls which exclude cellphones from their samples; at the moment, there does not appear to be sufficient statistical certainty to do so.
I’m sure that some people will also wonder what impact this adjustment has on Rasmussen’s polling, which has had a large, Republican-leaning House effect this cycle. The fact that Rasmussen has always sampled likely voters, whereas some other polls survey registered voters adults, accounts for some of its house effect. However, as we reported before, it does not account for all of it: we still show a Rasmussen house effect of 2-3 points even once this is accounted for.
Research 2000 polls removed. We no longer include any polls from Research 2000 because, in my considered professional opinion, the preponderance of evidence suggests that some or all of their data may have been falsified, or that if it was not falsified, it nevertheless reflects highly unorthodox and unscientific polling practices. Although I take no position on whether Research 2000’s polling has been proven to be falsified “beyond the shadow of a doubt”, this is not a court of law and we have no reason to apply that stringent a standard. If Research 2000 made even a rudimentary effort to provide us with the raw, interview-level data from its polls, or records of its transactions with its call centers, I would probably give them the benefit of the doubt, but so far they have not. (Note that I have not physically removed Research 2000’s polls from my database — they are still in there, but they are assigned a special flag that gives them a weight of zero. If Research 2000 were able able to make me reasonably comfortable that it in fact conducted the polls that it claims to have conducted, I would of course consider re-including them.)
Note, however, that in the case of Research 2000, this actually has a fairly minor impact, as their polls had already been assigned a low weighting because of their mediocre track record, and because they were already being substantially adjusted because of their large, Democratic-leaning house effect.
Use of partisan identification rather than PVI. Our forecasts are based, in part, on a regression model which seeks to predict the polling average in each state based on a number of variables, like the incumbent status of each candidate and their favorability or approval ratings. One of the variables in this analysis had been a state’s Partisan Voting Index (PVI), which is a measure of how it voted relative to the rest of the country in the Presidential elections of 2004 and 2008. However, growing out of some concerns with the numbers the regression model was producing in West Virginia, I decided to test an alternate measure of a state’s partisan orientation, which was the number of people who identified as being a member of each party in Gallup’s tracking throughout 2009. The partisan identification measure turned out to have substantially more predictive power than PVI (in fact, PVI adds no additional value once it is accounted for), so we will now be using party identification in PVI’s place. This helps the Democrats in states like Kentucky, which has relatively high levels of Democratic identification relative to their voting in the past two Presidential elections, but hurts them in places like Colorado, for which the opposite is true.
West Virginia now included. We now include a forecast for West Virginia, where Robert Byrd died last month and where there may now be a special election to replace him. We assign a 25 percent likelihood to each of the following scenarios: there is a special election between Joe Manchin and Shelley Moore Capito, there is a special election between Joe Manchin and Betty Ireland (these two matchups have been polled), there is a special election between a generic Democratic and a generic Republican candidate (the forecast for this scenario is based solely off of regression analysis), and there is no special election until 2012. Overall, the model figures that there is a 12 percent chance that the Republicans take over the seat in West Virginia, mostly based on the scenario where there is a special election but Manchin does not run.
*-*
As noted above, it is these methodological changes that account for most of the improvement in the Republicans’ standing in this month’s forecast. We actually show the national trendline — based on an analysis of generic ballot polling — having been exceptionally flat for at least the past 4-5 months, with Republicans having gained only about half a percentage point during that period. (This contrasts with Barack Obama’s approval ratings, which have shown some additional signs of erosion.)
Still, this is by all means looking like a very bad cycle for Democrats, and they are running out of time to right the ship. Republicans clearly have enough states in play to become the majority party. On the other hand, they are playing defense in a number of places, and some pickups that looked fairly certain before (Nevada, Delaware) look less so now. Therefore, they will probably need some additional national momentum of their own to have a clear shot of taking over the Senate.
Detailed results follow below. (Note: our earlier post this evening had incorrectly omitted a PPP poll of Wisconsin. We put the Republicans’ chances of taking over the state at 26 percent, not 24 percent. The other numbers in this article have been corrected accordingly.)
