In response to some comments by Brendan Nyhan and Alan Abramowitz, Nate writes about the generic congressional ballot, those surveys that ask randomly-sampled Americans a question like,
Looking ahead to the Congressional elections in November, which party do you plan to vote for if the election were being held today?
Nate’s remarks are reasonable and I just wanted to add a bit of clarification.
The bottom line is that, yes, you can use generic congressional polling to predict the national election results. But you don’t just take the generic ballot as is; you have to use it in stages, as part of a district-by-district forecast.
(1) Nate discusses the generic ballot (the percentage of survey respondents who say they would vote for the Democrat or the Republican) as a predictor of the actual ballot (the average share of the votes received by the two parties in the general election). I’d just like to emphasize that the prediction of actual from generic vote is done using a statistical model. For example, in October, 2006, Bafumi, Erikson, and Wlezien wrote: “Based on the current average of the generic polls (57.7% Democratic, 42.3% Republican) the forecast from this equation is a 55% to 45% Democratic advantage in the popular vote.” The shift from 57.7% to 55% came from a model that Bafumi et al. had fit to earlier congressional elections.
In particular, Bafumi et al. fit different models to predict the election outcome from generic ballot polls taken 300 days before the election, 240 days before the election, 180 days before the election, and so forth. One thing they found was that, when the incumbent president is a Democrat, the Democrats’ vote in the off-year congressional elections tends to be much lower than the generic polls taken 200-300 days before the election. The generic poll taken hundreds of days before the election is a good predictor of the ultimate outcome–as long as the prediction is made using a fitted model rather than merely by taking that generic poll number as is.
Here’s an example. In September, 2009, Chris Bowers wrote, “Republicans not in a position to retake the House (yet),” based on his observation that the Democrats had a 41-38 lead in generic House polling. But, having read Bafumi, Erikson, and Wlezien, I knew that Bowers was wrong, and I said so right here on fivethirtyeight: given the generic polls at that time, the best forecast was a 53-47 popular vote win for the Republicans.
(2) Nate writes, “the national House popular vote is an imperfect predictor of the seat count” and illustrates with a seats vs. votes scatterplot. He’s right, but this is less of a problem than you might think. The right way to do a congressional forecast is at a district-by-district level. The generic ballot (and other information) can allow you to predict the average district vote at the national level. And then, separately, you can predict the relative positions of the different districts (from most Democratic to most Republican), given district-level information (most simply, previous election results, corrected for incumbency, as in my papers with John Kastellec and Jamie Chandler from 2006 and 2008, or else maybe something more sophisticated using more up-to-date district-level information). Since Nate is doing district-by-district forecasting anyway, this isn’t a problem.