As many people have noticed, Rasmussen’s polls this year have tended to be more favorable to Republican candidates and causes than those of most other polling outfits.
This does not necessarily imply bias. It could result from a whole number of things, ranging from the fact that they’re applying a likely voter screen when most other pollsters aren’t, to some sort of methodological quirk, to the fact that they’re just doggone right.
But the degree of the “house effect” seems to vary according to the nature of the candidate. When the candidate is a mainstream or liberal-ish Republican, like the candidates in Missouri or Ohio or Connecticut or Illinois, Rasmussen’s numbers don’t seem to differ all that much from other pollsters; maybe a couple of points, on average. But if the candidate is more of an “insurgent” conservative with ties to or roots in the tea-party movement, you can see some really big differences.
Take, for example, their recent polls in Kentucky and Florida. In each case, you have someone who gets the tea-partiers excited (Rand Paul and Marco Rubio, respectively) running against a more traditional mainline Republican.
The other pollsters, for the time being, have the mainline Republican doing better; in Rasmussen’s case the opposite is true. In Florida, Quinnipiac has Charlie Crist outperforming Marco Rubio by a net of 23 points against the Democrat, and Research 2000 by 25 points. But Rasmussen has Rubio doing 8 points better. A similar trend manifests itself in Kentucky for Rand Paul in his matchup against Daniel Mongiardo, although is not so apparent against the other Democrat, Jack Conway.
I’ve also included the percentages of voters in each survey who had a strong enough opinion about a candidate to rate him — this is not exactly the same thing as name recognition, but it’s pretty close so we’ll use that term as a shorthand. Research 2000 and Quinnipiac have Rubio with name recognition of 43 percent and 38 percent, respectively — Rasmussen has him at 80 percent. PPP and Research 2000 have Rand Paul with name recognition at 49 and 55 percent; Rasmussen has him at 81. (EDIT: To be fair, some of this is because Rasmussen’s polls are somewhat more recent — they had Rubio at 71 percent name recognition in August and 72 percent in October, and Paul at 74 percent in September.)
This, at least, helps to explain why their results are different: the voters in their survey are much more likely to be tuned in and to have heard of candidates like Rubio and Paul, and are therefore more likely to say that they’ll vote for them. Rasmussen, I’m sure, would say that this is because they’re using a likely voter screen whereas the other pollsters are not.
But personally, I just don’t buy that 80 percent of the people who will eventually vote in November have an opinion about Rand Paul or Marco Rubio right now. These candidates are relatively fresh faces in their respective states; the political class may know about them, but most voters (even “likely” ones) aren’t going to tune in to the horse race stuff for several more months and may have at best a passing knowledge of the candidates at this stage.
Moreover, the differences in name recognition are really large. Suppose that Quinnipiac is right that Rubio has 38 percent name recognition among registered voters. Suppose moreover that all of the people who have heard of Rubio are people who are going to vote in November and are properly identified as likely voters.
Turnout in midterm elections is usually in the range of 60 percent of registered voters, and will probably be somewhat higher than that in Florida where there are two highly competitive races. But let’s say it’s 60 percent, and you can identify this 60 percent with 100 percent accuracy. If that’s the case, the highest Rubio’s name recognition should theoretically be in your poll is 63 percent, which is 38/60. But Rasmussen had him at 71 percent two months before the Quinnipiac poll was conducted, has him at 80 percent now. Some of this could result from differences in question wording; Quinnipiac, for instance, explicitly gives the respondent a choice of “or haven’t you heard enough about him?”. Nevertheless, the Rasmussen respondent always seems ready to take a position on just about everything, including candidates and issues that other pollsters show he doesn’t know very much about. This perhaps reflects a form of Type II error or “false negatives” — leaving people out of your sample who should be included, specifically people who are less well informed but who do vote.
Although it seems likely that Rasmussen is winding up with an over-tight sample of voters, I don’t think that they’re doing so intentionally. If you’re a robopollster, you’re lucky enough to get someone to complete your survey 10 months out from a Congressional election when call screening is becoming more and more common and people have a lot of other things — notably the economy — to worry about. And, unlike the other robopollsters like SurveyUSA and PPP, Rasmussen does all their polling in one evening, so if they miss you the first time around, you’re out of their sample. Their response rates have to be very low. If someone has picked up the phone and completed your survey, that’s probably all the indication you need that they’re engaged by politics and they’ll probably vote come November.
But then Rasmussen applies a likely voter screen on top of that. This is probably overkill. If you’re Quinnipiac, and you have a huge academic budget and a whole room full of trained professional interviewers and are in the field for five or six days, then you can and perhaps should be applying a likely voter screen, even at this early stage. If you’re Rasmussen and doing a blitz poll, your voters have pretty much already been screened for you. It’s even possible that your sample is already too tight before the likely voter screen and that you’d theoretically want to make it looser, although there’s really no good way to do that.
Weighting by party ID, as Rasmussen does, gives a pollster a pretty big fudge factor, but only provided that the response biases in your sample come between the categories that you’re weighting by and not within them. Rasmussen’s weighting scheme might guarantee that they have more or less the right number of Republicans or the right number of Hispanics — but it can’t guarantee that they’ll have the right kind of Republicans or the right kind of Hispanics, particularly for amorphous factors like ideology which can be hard to proxy. It’s possible, for instance, that they’d wind up with Republicans who were too informed and/or too conservative relative to other Republicans. If that’s the case, it would probably manifest itself in the way that certain Republican candidates performed vis-à-vis other ones, as we may be seeing in their polling of Kentucky and Florida.
By the way, this is not necessarily an indictment of the ultimate accuracy of Rasmussen’s polls. This “bug” may even be helpful to them! It’s quite possible the uber-tuned-in voters that they’re sampling are “early adopters”, and the rest of the electorate will follow their lead. But in that sense, their polls are more like predictions and less snapshots of current public opinion.