Last week, Opinion Research Associates released a poll in Arkansas putting Democratic Sen. Mark Pryor 10 percentage points ahead of his Republican challenger, Tom Cotton. Other recent polling has had the race far closer, with Pryor leading by just a few points. On Tuesday, a poll released by the Democratic Senatorial Campaign Committee (DSCC) showed Pryor up by 3 points.
It would be easy to dismiss the Opinion Research Associates poll — conducted on behalf of Give Arkansas a Raise Now, a group pushing to increase the minimum wage — as an outlier. But as I’ve noted recently, there hasn’t been a lot of polling in Arkansas. How much should we discount last week’s survey because it was partisan?
FiveThirtyEight has reported that partisan polls for the House have had a bias of about 6 points in favor of the party associated with the sponsor of the survey. But what about the Senate?
Our polling database has 147 publicly released partisan Senate polls since the 2002 election. (Disclaimer: The database is almost certainly missing some surveys.) I looked at how partisan polls conducted in the final month of the campaign compared to election results.
Partisan polls since 2002 have had an average bias of 4.25 points in favor of the side who sponsored the survey. For example, a survey sponsored by a Democratic group that showed the Democratic candidate ahead by 3 points would have an error of 5 percentage points if the Republican won the election by 2 points.
In our sample, Republican-sponsored polls were no more accurate or less accurate than Democratic-sponsored surveys.
Only 16 percent of the time did a partisan Senate poll underestimate its side’s candidate by a percentage point or more. Only 4 percent of the time was the underestimation 5 points or more. Only 2 percent of the time was the error 7 points or more, which is the difference between the DSCC and Opinion Research Associates poll.
We’re most likely to see the greatest bias when a candidate is losing by a lot and a partisan poll is released from that candidate’s side. The average bias when the sponsored side loses is 6.64 points. It averages only 0.98 points when the sponsored side wins. When the sponsored side wins by more than 5 points, the bias is nonexistent.