Skip to main content
ABC News
Canaries in the Coal Mine? Or Cuckoo Polls?

Raúl M. Grijalva. Bennie Thompson. Jan Schakowsky.

Jim Oberstar? John Larson? Corrine Brown?

Peter DeFazio! John Dingell! Barney Frank!

These are some of the Democratic incumbents, most long believed to be safe by analysts. But in recent weeks, polls of their districts have suggested surprisingly competitive races.

Some of these polls show the Republican challengers leading; most show the Democrat ahead, but by a smaller margin than expected. Some of the polls are from independent researchers; most are issued by the campaigns themselves, or by other Republican-affiliated groups. Some of the companies conducting these polls have strong reputations; others have little track record.

What these polls have in common, however, is that each time one is issued, they make Republicans very excited — and Democrats very nervous. It’s time for a more sober look at them.

I ran a scan of our database for polls issued since Labor Day in districts where there is a Democratic incumbent who won with at least 65 percent of the vote in 2008, or did not face a Republican challenger at all. I identified all polls in these districts that showed the Republican within 10 points of the Democrat, or with a lead of any margin.

There were a total of 22 polls, among 18 congressional districts, that met these criteria:

That chart is pretty dense with information, so let’s sort through it:

First, relatively few of these polls actually show the Republican candidate ahead. And in each of the districts where there is an exception — the Illinois 17th, the Michigan 15th, the North Carolina 7th, and the South Dakota At-Large district — there is at least one other poll showing the Democrat with the lead.

Second, while each of these Democrats breezed to re-election in 2008, not all of them are in districts that are inherently all that friendly to Democrats as measured by Partisan Voting Index, which measures the preferences of voters in presidential elections. While some, like Mr. Frank and Mr. Dingell, are in districts where Barack Obama received an overwhelming majority of the vote in 2008, others like Mr. DeFazio are in districts closer to the national median. And some, like Gene Taylor of Mississippi, are in very Republican-friendly ones. On average, these districts had a Partisan Voting Index of Democrat +2, making them just slightly bluer than the country as a whole.

Third, many of the polls are either partisan-affiliated, or were “robopolls” that used automated scripts rather than live interviewers, or both. (The former are highlighted in red in the table; the latter are given the designation “I.V.R.,” for interactive voice response).

Polls with an explicit partisan affiliation are on average about 6 points friendlier to their candidate than those conducted by independent groups. Robopolls have not shown any persistent bias in the past — but this year, they have been 2 to 4 points more favorable to Republicans than traditional surveys, and the differences have tended to be larger in polls of House races as opposed to conducted in Senate or gubernatorial campaigns.

So this is a group of polls that you’d expect to be pretty Republican-friendly. The only polls that were both traditional and nonpartisan were in Indiana’s 2nd District, where EPIC/MRA put the Democratic incumbent Joe Donnelly 9 points ahead (a  decent enough result for him), and in the Mississippi 2nd.

FiveThirtyEight’s forecast for each district is listed in the rightmost column. It sees several of these races as being tossups, and in one case, makes the Republican, Kristi Noem of South Dakota, the outright favorite. In other cases, however, it gives the Republican less than a 10 percent chance in spite of the poll — and in some instances, gives the Republican less than a 1 percent chance.

These forecasts should be approached with some caution. In contrast to our Senate and gubernatorial forecasts, our House model looks at local polls in connection with a whole host of other variables. In particular, the model relies heavily on expert forecasts, like those issued by Cook Political and CQ Politics. If these experts still regard the district as “safe” or “likely” Democrat, the model will generally not give the Republican much of a chance, as candidates within these groups have very rarely lost their elections in the past.

On many occasions, these experts will have access to other information — like private polling in the district — that will give them good reason to be dismissive of the publicly-released surveys.

But there are also times when these forecasts can be lagging rather than leading indicators. And undoubtedly, they have their blind spots. In particular, our research suggests that they tend to give too much weight to an incumbent’s vote share in previous elections, and not enough to the underlying characteristics of the district, which may be better represented by how it voted in presidential elections. An incumbent who coasted to re-election in past years may have done so simply because he was facing an underwhelming opponent who didn’t run a fully-fledged campaign. Against a competent and well-financed challenger, however — and Republicans have those in plenty of districts this year — he may not find the terrain so friendly.

Nevertheless, I don’t think these polls ought to be quite as scary for Democrats when placed into proper context. It is tempting to think that these polls represent the canary in the coal mine: If incumbents like Mr. Frank and Mr. Dingell are in competitive races, then surely many others must be, too?

But that may be exactly the wrong way to look at these polls. There are surely dozens of districts where the Democratic incumbent is in fact perfectly safe for re-election: he may be 20, or 30 or 40 points ahead. But we aren’t likely to see any polling in these districts. It is very rare to see a candidate release an internal poll to the public when he’s more than about 15 points behind — what’s the point? It is also very rare to see a candidate release a poll when she is more than 15 or 20 points ahead, the only effect of which might be to generate complacency.

This is not to suggest that Republicans don’t have enormous upside in these elections. If they win 60 or 70 seats — outcomes we believe are plausible — they are going to win a few seats that were on nobody’s “list.” Their victims might include some of the incumbents mentioned above — or others that nobody had given much thought to.

Even with a smaller gain, like 45 or 50 seats, Republicans are likely to pull off a couple of major upsets. The same is also true of Democrats, by the way. Even if they are having a poor night over all, a few of their candidates who seem extremely vulnerable will somehow defy the odds — or they may knock off a Republican incumbent or two in places where they weren’t favored to. That’s what happens when you have 435 different elections taking place in every place of the country.

But I don’t think these polls provide much information about the political climate that isn’t already apparent from other indicators. If Democrats are bound to lose about 50 seats — and if Republican campaigns are very aggressive about releasing internal polls that show them at all close, as they have been — then this is probably about what you’d expect too see on a “greatest hits” list. Likewise, you could also create a highly curated list of polls that show Democrats in a surprisingly strong position — there are plenty of those, too.

Our focus, instead, remains on the big picture — and that picture is scary enough for Democrats.

Nate Silver is the founder and editor in chief of FiveThirtyEight.