Skip to main content
ABC News
Aug. 2: When a Poll That Seems Like News Isn’t

On Thursday around 4 p.m., my Twitter feed lighted up with people either promoting or critiquing a new national poll from Pew Research that gave President Obama a lead of 10 percentage points, 51 percent to 41 percent, over Mitt Romney.

It’s an impressive number for Mr. Obama on the surface — but when taken in context, the poll doesn’t tell us anything all that different about the presidential race.

The context is simply that Pew has consistently shown very good numbers for Mr. Obama so far this cycle. In the seven national polls they released before this one, dating back to January, Mr. Obama led Mr. Romney by an average of 51 percent to 44 percent. A poll showing him ahead by 51-41 instead obviously isn’t all that different.

Our forecast model is designed to account for a pollster’s prior take on the race in a few different ways, but the most important is its calculation of a house effects adjustment, which measures how a given polling firm compares with the consensus of surveys. In the case of Pew Research, our method estimates that their polls have been a net of 3.8 percentage points more favorable toward Mr. Obama than the pollster consensus.

Our model essentially just subtracts out that house effect from the polling firm’s top-line results. So instead of treating the poll as a 10-point lead for Mr. Obama, it instead treats it as a 6-point advantage.

But wait, there’s more! The Pew poll was a survey of registered voters, and those numbers are typically more favorable to Democrats than polls of likely voters. Our model adjusts for this as well. Historically, Republicans gain a net of 1.5 percentage points in the transition from registered voter to likely voter polls, and our method now estimates that the gap will be slightly larger than that — 2.3 percentage points — this year.

So subtract roughly another two points from Mr. Obama’s lead. From the forecast model’s standpoint, therefore, the Pew poll is treated as a four-point lead for Mr. Obama after making these adjustments.

A four-point lead for Mr. Obama is a decent enough number for him, but nothing that should drastically change our understanding of the race.

In fact, Mr. Obama’s forecast actually fell slightly on Thursday, although not by a significant margin, to a 70.2 percent chance of winning the Electoral College from a 70.8 percent chance on Wednesday. If the Pew poll was a good data point for Mr. Obama (if not as good as it appears at first glance), he was negatively affected by a poor poll in North Carolina, and by the decline in the stock market.

Pew is a pollster I have a lot of respect for, by the way. But polling elections is just not that easy in an environment where even the best pollsters get only about 1 in 10 people to complete their surveys. Gallup is another pollster I have a lot of respect for, yet their polls have been a few points Republican-leaning so far this cycle, relative the pollster consensus.

Accounting for these house effects, and the impact of registered voter against likely voter polls, can potentially make your forecast average a lot more stable — and save you a lot of heartburn when the numbers seem to fluctuate up and down every day.

Our methods, which adjust for these effects, and which also uses state polls along with national polls to make inferences about the state of the race, has tended to show a steadier race than methods like the RealClearPolitics average. The RealClearPolitics method is highly useful to consumers precisely because of its simplicity and transparency. But, nevertheless, it sometimes conveys the sense that there is a change in the state of the race when there probably isn’t, just because a pollster with a pro-Obama or pro-Romney lean may happen to enter or exit the polling average on any given day.

To be sure, there are cases when a day’s worth of polling can have a material impact on the forecast. Wednesday was such a day, for instance. What you had on Wednesday was several different polls of several different swing states (Ohio, Florida, Pennsylvania and Michigan), all of which were pretty favorable to Mr. Obama.

And just as important, all of them were from polling firms that had not shown especially good results for Mr. Obama previously.

Conversely, that the Pew shows a big lead for Mr. Obama is good news for him, but it was news that was already priced into the forecast.

Nate Silver founded and was the editor in chief of FiveThirtyEight.