Those of you who are as obsessive as we are about checking for new Senate polls might have noticed a change in the FiveThirtyEight Senate forecast interactive. We’re now listing the polls the FiveThirtyEight model uses in each state, along with how the model weights and adjusts them.
Here’s what the data looks like for Colorado, for example:
We’ve listed both the raw and adjusted margin associated with each poll. As we describe here, the model adjusts polls for a few reasons. If a survey is conducted among registered voters (or all adults) rather than likely voters, the model will shift its results toward Republicans because likely voter polls tend to show better results for the GOP (and also tend to be more accurate). However, almost all polls being released now are conducted among likely voters, so this has little effect in most states. Polls are subject to a trend line adjustment based on changes in the generic congressional ballot, but this also has little effect.
Instead, most of the difference between the raw and adjusted figures has to do with our house effects adjustment. Some pollsters consistently show more favorable results for one party or the other compared to other polls. Quinnipiac University polls, for example — although they have a very good track record — have had somewhat Republican-leaning results this year compared with the consensus. So the FiveThirtyEight model treats the Quinnipiac poll that showed Republican Cory Gardner up by 8 percentage points as closer to a 6-point GOP lead instead.
You can also see the weight the FiveThirtyEight model assigns to each poll. Keep in mind: The weights are based on sample size, recency and a firm’s pollster rating. In Iowa, for example, the recent Public Policy Polling survey is weighed about as much as the recent Des Moines Register (Selzer & Company) survey. Although Selzer has a much better pollster rating, PPP took a considerably larger sample; in this case, those two factors roughly cancel out.
If you don’t see a poll listed, it may be because it’s a poll conducted directly on behalf of a candidate or party committee; FiveThirtyEight does not use these polls in its forecast. (We do use polls from interest groups.) Or it may be that we’ve missed the survey, or it’s too new to have been added to our database. Please drop us a line if you think you’ve found a poll we’ve missed.
On the right-hand side of the polls page, you can see the steps the FiveThirtyEight model takes in making its forecasts. Here are those steps for Colorado, for instance.
In this case, the Democrat (Mark Udall) benefits slightly from the polling adjustments because the recent polls in Colorado have been a slightly Republican-leaning bunch (according to our house effects calculation). But this isn’t true in every state.
“State fundamentals” reflects a regression-based estimate of the election result. The factors included in the estimate are described in detail here and include measures of state partisanship, candidate ideology scores, generic congressional ballot polls, fundraising totals and others. In Colorado, the fundamentals calculation has Udall slightly favored instead of slightly behind.
The fundamentals receive little weight in well-polled states like Colorado. Specifically, the model assigns them a weight of 0.35. In Colorado, the polls have a cumulative weight of about 6. That means the polls make up about 95 percent of the forecast.
Finally, the model makes a distinction between the “now-cast” — what we project would happen in an election held today — and the projected Election Day vote shares. This reflects the historical tendency for midterm elections to favor the party that doesn’t occupy the White House. This adjustment is conservative to begin with and then phases out completely by Election Day. Because we’re into October, it has almost no effect now.
Have fun browsing, and let us know if anything looks awry.