So, whom to believe?
The special election to replace Ted Kennedy in Massachusetts has become quite intriguing. And three pollsters with surveys out this week all have rather different takes on the race.
The Rasmussen poll shows the Democrat, Martha Coakley, up by 9 points. The Boston Globe poll, conducted by the University of New Hampshire survey center, has her up by 15. But the PPP poll actually has Scott Brown up, by 1 point. All three polls are of likely voters.
In addition to these, there’s also a rumor of a Boston Herald poll that shows Coakley up by just 1, a supposed leaked Republican internal poll that had Coakley up by 11, and some older polling (including one of likely voters) that showed Coakley up by margins ranging from 26 to 31 points. But we’ve already got our hands full with Globe, PPP, and Rasmussen, so let’s focus on those for now.
All three polling firms in question have above-average track records. PPP and UNH have a history of the occasional big miss, although they usually get things right. Rasmussen tends to be a bit more consistent, although their polling has been a bit “different” since the 2010 cycle began — they may turn out to be dead-on about 2010, but until we know for sure, I might apply a bit more skepticism to their results than I would ordinarily.
The PPP is more recent than the other two, although only by about 48 hours. Recency is nice, but unless there’s a specific major event that drives changes in public opinion (not just general “momentum”), I tend to think it’s a bit overstated. Plus, there’s a downside to recency; half the PPP sample was conducted on Friday, a day that some pollsters like to avoid.
The PPP poll has a bit larger sample size than the other two; 774 respondents versus 554 and 500, respectively.
Rasmussen’s partisan ID breakdown is a bit less plausible than the other two, although as the Globe poll makes clear, there’s a big difference in Massachusetts between partisan registration and partisan identification, which makes this somewhat ambiguous.
The Globe poll explicitly mentions the name of the third-party candidate, the libertarian Joe Kennedy, and gives him 5 percent of the vote. The Rasmussen poll does not mention him by name, but provides a choice for “some other candidate”, who gets 1 percent. And PPP does not provide for a third party option at all. You can make a case either way here; although Kennedy is participating in the debates and getting a bit more attention than usual, there’s also some history of polls overstating the margins that third-party candidates receive on Election Day.
Globe/UNH is the local pollster, whereas Rasmussen and PPP are national ones. Rasmussen has nevertheless polled Massachusetts rather frequently before, although PPP hasn’t.
Rasmussen and PPP use the IVR method — a.k.a. “robopolls” — whereas Globe/UNH uses live interviewers. But there’s not really convincing evidence about whether IVR polls are inferior to regular ones. And in some recent elections, in fact, they’ve tended to outperform them. They do get lower response rates, however, which can at least potentially raise questions about response and self-selection bias.
* * *
Are you seeing a consistent pattern here? I’m not. All of the polls have positives and negatives. And any of them could be right.
The average of the three polls shows Coakley up by 8 points. As I’ve written before, I would probably take “over” on that 8 percent number. Fundamentally, this is still Massachusetts, and unless the Democratic candidate has some sort of fatal flaw (Coakley is a bit boring, but that’s hardly an unpardonable sin), it’s just going to be a really heavy lift for the turnout to be lopsided enough to allow the Republican to prevail.
At the same time, while I’m taking the “over” on that 8 percent, I’m also taking the over on variance. Special elections are notoriously hard to predict. And we also seem to be at some weird sort of inflection point in the electoral cycle. You can point toward some evidence to make the case that the bottom is really falling out from Democrats, and you can point toward other evidence which suggests that the whole tea-party backlash, while not unimportant, is really just operating at the margins. So, I acknowledge that there is a fairly tangible shot of Brown winning — higher than the 3-5 percent I assigned to him after seeing the Rasmussen poll, but lower than the 15-25 percent chance I gave him before seeing the Boston Globe result.
This probably won’t be a very satisfying answer to those of you who come to me looking for some kind of certitude. But part of being a good forecaster is knowing when to make a bold forecast and knowing when to proceed with more caution; the Massachusetts race calls for a heavy dose of the latter.
Edit: See also Mark Blumenthal, who has a similarly balanced take.