I mentioned in conjunction with yesterday’s Senate polling update that “house effects” — systematic differences in the the way that a particular pollster’s surveys tend to lean toward one or the other party’s candidates — have tended to be extremely strong thus far this cycle. Here, in fact, is a demonstration of the house effect that my model is identifying for each of the five most prolific pollsters so far this year — those that have produced at least 15 separate surveys for head-to-head Senate race matchups. (Our methodology for calculating these is described here.)
Rasmussen, thus far, has a Republican-leaning house effect of about 5 and 1/2 points. So if Rasmussen, for example, has a Republican leading by 7 points in a particular race, an average pollster would have the Republican ahead by only 1 or 2 points. Research 2000, on the other hand, has a Democratic-leaning house effect of about 4 1/2 points. If they show an R+7 in a particular race, it would be the equivalent of an R+11 or an R+12 from an average pollster. These are, obviously, very large differences: it implies that if Research 2000 and Rasmussen were to poll the same race, we’d expect about a 10 point difference between them.
It’s extremely important to emphasize that just because a pollster has a house effect doesn’t mean that it’s wrong. They may result from legitimate differences of opinion about how to conduct surveys. Rasmussen, for instance, is one of the few pollsters to already be employing a likely voter model at this point. It’s not uncommon for likely voter polls to have comparatively better results for Republicans, since Democrats rely on votes from groups like young voters and minorities who turn out less reliably in midterm elections. (And, indeed, Republicans appear to have an especially significant enthusiasm advantage in this cycle.)
What it does mean, though, is that when you’re evaluating the polls in a particular race, you need to take an especially long look at who’s conducting them. If you tell me that the latest poll has the Republican up by two points in Colorado, that’s going to mean one thing if it’s Research 2000 telling me that and quite another if it’s Rasmussen. Unfortunately, just a couple of pollsters have been very dominant this cycle. Rasmussen and Research 2000 alone collectively account for over half of the polls in my Senate database, and the fraction jumps to 67 percent if you include Public Policy Polling (which hasn’t had much of a house effect) as well.
By the way: when I publish something like this, my intent is not to chide the various pollsters into some kind of orthodoxy — I frankly like the difference of opinion! But it’s been a large difference of opinion so far this year, and if it carries forward to Election Day, the pollsters might have almost as much on the line in November as the candidates.