Henry Olsen at National Review Online has a smart critique up on my observation that the generic Congressional ballot may underestimate Democrats’ standing in House races this year. Let me warn you up front: this post is going to get into some fairly technical issues.
The most important question raised by Mr. Olsen’s article actually doesn’t have to do with the generic ballot per se, but rather boils down to whether polls of individual races tend to underestimate the standing of candidates who have relatively poor name recognition (as is often the case when, for instance, a non-incumbent is matched up against an incumbent in a U.S. House race). This is a real effect, and Mr. Olsen is right that it is something to be mindful of. Early in an election cycle, in fact, it’s something to be very cognizant of — candidates who are poorly known to voters generally have some upside in their numbers. By the time we reach Election Day, however, the bias against these candidates pretty much disappears — if nobody knows who you are by the morning of the election, well then, it’s probably too late.
Our House forecasting model has some ways to account for this: for instance, it tends not to look very much at polls of individual districts early in the election cycle, but tends to place more emphasis on them (at the expense of the generic ballot) as Election Day draws nearer. Also, looking at the number of undecideds in a poll can sometimes be informative: a 40-30 lead in the polls is not as solid as a 50-40 lead.
I owe Mr. Olsen a longer response on some of these points (I actually don’t think we disagree on very much). Perhaps more important, I owe FiveThirtyEight readers a more comprehensive overview of our House model in general.
But, for the time being, I want to focus on one particular comment that Mr. Olsen made. He writes:
But, as one noted prognosticator observed earlier this year, “On average the generic ballot has overestimated the Democrats’ performance in the popular vote by 3.4 points since 1992.” When this is applied to the AAF data Nate cites, it appears that the Democratic problem is not better than it appears; it’s worse.
The “noted prognosticator” that Mr. Olsen refers to is yours truly! As I wrote in April, there’s some history of the generic ballot overestimating the Democrats’ performance in the national House popular vote.
The House popular vote is what you get if you simply add up the votes for the Democratic and Republican candidates, respectively, across all 435 congressional districts. For instance, in 1998, Democratic candidates received a total of 31,490,298 votes for the House, while Republicans received 32,233,067, and candidates from other political parties, 2,154,221 votes. That translates into a Republican win of about 1 percentage point in the national popular vote. Most of the generic ballot polls that year, by contrast, had shown Democrats with a slight advantage, so this was one of those years in which it somewhat overestimated their performance.
This is only relevant, however, to the extent that you care about the aggregate House popular vote — which you might, for instance, if you were using the popular vote to back into an estimate of the number of seats that a particular party might gain or lose. (That’s what I was trying to do back in April.) If you’re working with this type of model, you have a decision to make about whether to apply a correction for the fact that the generic ballot has tended to overestimate the Democrats’ popular vote performance in the past. (For a variety of reasons — like the fact that the effect seems to have become less profound in recent elections, and that other types of polls haven’t shown a systematic bias toward one or the other party — it’s not quite so straightforward a decision as it seems, but it’s certainly something a forecaster has to wrestle with.)
That is, however, not the type of model we’re working with now, mainly because it isn’t very precise. Instead, we’re working from the ground up, trying to make calls on each of the 435 individual House races, and then aggregating those projections in a careful way to figure out how many seats each party is likely to control overall. So what we’re really concerned with is what information the generic ballot can provide is about how candidates will perform in individual districts.
Let’s break to look at some data. The table below contains four numbers for each election cycle since 1994.
- First, the final trendline estimate among all generic ballot polls of likely voters.
- Second, Gallup’s final likely voter estimate of the generic ballot (a number of forecasting models are based on Gallup’s poll in particular)
- Third, the national House popular vote, as described above.
- And fourth, the average margin separating the two parties over all 435 Congressional Districts in each election.
There are a few things to notice here. First, as I noted in April, most generic ballot polls have tended to overestimate the Democrats’ performance on the House popular vote in most years.
Second, Gallup’s likely voter poll is close to being an exception — it has tended to show worse results for Democrats than other likely voter models, and there is little evidence that it has any systemic bias.
Another finding is that Democrats do better in the average district than in the aggregate popular vote. Since 1994, they have won the average Congressional district by 3.3 percentage points — even though in the aggregate popular vote, they’ve been essentially tied with Republicans.
This sounds like a paradox, but the reason for it is that Democratic-friendly districts — which often have more minorities and young voters — usually have lower turnout than Republican-friendly ones. Compare a district in which the Republican wins by 10 points and 500,000 people vote, to one in which the Democrat wins by 10 points and 300,000 people vote. Between these two districts, the Republicans will have received more total votes, since the Republican win will have come on heavier turnout. But the margin between the two parties in an average district is zero. This is similar to what happens in House elections in the real world.
While the generic ballot has tended to overestimate the Democrats’ aggregate popular vote, it has not tended to overestimate their average vote in individual Congressional districts — or at least not to an extent worth worrying about (and Gallup’s poll has tended to underestimate it). Since our model operates at the level of individual districts, this convenient property allows us to take the generic ballot at face value: if it shows the Republicans ahead by 7 points, the model takes this as evidence that they will win the average Congressional district by exactly 7 points.
But the generic ballot is not the only piece of information that the model looks at — it also looks at a whole host of others, like district-level polling. These other indicators — while still quite weak for Democrats — have been notably better for them this year than the generic ballot. This, then, is the basis for the claim that the generic ballot may underestimate Democrats’ standing.
Of course, there is an alternate hypothesis: it’s not that the generic ballot underestimates Democrats, but that the other indicators — like local polls — overestimate them! That would do just as well to explain the discrepancy.
Our forecasting model does not discriminate against this hypothesis at all — in fact, it believes it to be true. The model assumes that the generic ballot probably underestimates the Democrats a bit, and also, that the other indicators probably overestimate them a bit. It splits the difference between them. This happens to be the way that you would have gotten the best results in forecasting individual races in recent election cycles: all of these indicators have their problems, and you would have done best to aggregate them together in some form.
I sometimes get annoyed by analysts who concentrate solely on the generic ballot, at the expense of all the other indicators. Mr. Olsen is not guilty of this — he’s clearly thought very carefully about the advantages and disadvantages of different types of indicators. But some analysts tend to fetishize the generic ballot, probably because plugging it into a formula is just a lot less work than having to figure out, say, how to translate a Cook rating into the probability that a candidate wins a seat, or under what circumstances local polls are relatively more reliable and when they are less so.
I’m not fond of providing bullet-point summaries, but it’s probably worth doing so here.
- In recent elections, the generic ballot has tended to overestimate the Democrats’ performance in the aggregate House popular vote. Models that use the popular vote as one of their inputs may want to account for this.
- However, in recent elections, the generic ballot has not materially overestimated (nor underestimated) the performance of the Democrat in the average congressional district.
- In this election, there is a discrepancy between the generic ballot and other indicators, with the generic ballot tending to be comparatively less favorable to Democrats and the other indicators comparatively more favorable to them. This could indicate that the generic ballot underestimates Democrats, that the other indicators overestimate them, or some of both. Our model assumes some of both.