There are certainly some good reasons to think that the polls could break toward Mitt Romney. For instance, many polls out now were conducted among registered voters; when pollsters switch over to likely voter polls instead — which assess each voter’s probability of actually casting a ballot on Nov. 6 — it is likely that Mr. Romney will gain a point or two. And Barack Obama obviously has a lot of weight to bear from the lukewarm economic recovery.
But one hypothesis you should find less persuasive is the notion that the polls will break toward Mr. Romney just because he is the challenger. It is often asserted that this is the case — that the polls move toward the “out-party” candidate rather than the incumbent. But in my view the empirical evidence — although it is somewhat ambiguous — mostly argues against this idea.
For a bit of background on this question, see my previous work on the “incumbent rule,” which is the notion espoused above that polls tend to break toward the challenging rather than incumbent candidate. My research into polls of gubernatorial, Senate and House races over the last 15 years or so suggests that this just is not true; in general, the incumbent candidate has been as likely to overperform his polling margin on Election Day as to underperform it.
But what about in presidential elections?
The table below contains some data compiled from the past 11 presidential elections — 1968 through 2008 — which is when the polling becomes more robust in our database.
What I have done here is to take a monthly polling average for each month in the year leading up to these past elections — for example, the average of all national polls conducted in April 2000. Then I have averaged these monthly averages so that each election year gets the same weight in the analysis and placed them into two groups: the polling average for the incumbent party’s candidate and for the challenging candidate.
The data point for April, for instance, shows the incumbent candidate at 44.6 percent and the challenging candidate at 41.5 percent. What that means is simply that, on average during election years since 1968, the incumbent party’s candidate had 44.6 percent of the vote in polls in April and the challenging candidate 41.5 percent.
Finally, I have compared the poll averages against the actual outcome. The results account for only those election years in which there were polls available in that month. We have no Nixon-McGovern polls from February 1972 in our database, for instance, so 1972 is not included in the actual results column for February, just as it is not included in the polling column. This allows for an apples-to-apples comparison on how both the incumbent-party and challenging candidates performed vis-à-vis the polls.
What do we see here? For the most part, the incumbent-party and challenging candidates get a higher share of the vote than they do in the polls, which simply reflects the fact that some number of undecided voter will migrate into their camps.
Perhaps more important, both candidates gain about the same amount of ground relative to the polls. On average since 1968, for instance, the incumbent-party candidate gained 3.5 points between the September polls and his actual performance on Election Day, while the challenger gained 3.9 points.
But, there are a couple of clear exceptions to this rule.
First, the challenger has considerably overperformed polls conducted early in the race — in particular, those polls from about December in the year before the election until March of the election year.
What’s going on here? I suspect that this has to do with the primaries — that December-through-March period is the time when the challenger is typically engaged in heated competition against his intraparty rivals for the party’s nomination. It is hard to appeal to general-election voters at the same time you are hoping to assert your credentials with your base — and your general election polls will usually suffer for it.
We saw this pattern this year, for example. Mr. Romney went through a profound slump in the general-election polls in February and March, during which time Mr. Obama’s lead averaged about 5 points — larger than it did before or after. These were the months when Mr. Romney was having the toughest time in the primaries, sometimes even trying to prove that he was to the right of either Newt Gingrich or Rick Santorum on crucial issues. But Mr. Romney’s polls snapped back into place once he effectively locked up the nomination.
Another factor during this early period may be name recognition. There are some people who (quite reasonably) will not be willing to tell pollsters that they will vote for a candidate whom they have never heard of. And the challenger is typically much less well known than the incumbent-party candidate, who is normally the sitting president or vice president.
This name-recognition factor might not have been as much of an issue for Mr. Romney as for some other candidates. Mr. Romney ran for president in 2008 and many voters remembered him, so his name recognition was already on the order of 75 percent as of early last year. But some candidates in past years — like Bill Clinton and Jimmy Carter — have emerged from relative obscurity to win their party’s nomination, and their polls may suffer until they become more of a household name.
Once a candidate wins the nomination, however, he will almost certainly be universally known, as Mr. Romney now is. (Mr. Romney’s name recognition is now about 95 percent, and the few Americans who do not yet know his name are presumably deeply disconnected from politics and are unlikely to vote in November.)
The other anomaly has been in the summer months — June, July and August. In these months, the polls have tended to low ball the position of the incumbent party’s candidate. In July polls, for instance, the incumbent-party candidate has trailed by an average of 4 points in elections since 1968. However, the incumbent-party candidate has actually won these elections by an average of a little more than 3 points. In other words, there has been a seven-point swing on average toward the incumbent party’s candidate from the July polls until November.
What accounts for this? It may be the effects of the party conventions, which can produce sharp — but ultimately ephemeral — “bounces” in the polls toward the party holding its convention. The challenging party always holds its convention before the incumbent party, and until the most recent years, it often held its convention in July. So it gets its bounce first, making its numbers artificially high for a
month or so.
I do find that there is some tendency for the incumbent-party candidate to overperform his summer polls even after attempts have been made to account for these convention effects — but accounting for the conventions reduces that quite a lot. Whatever residual effects there are probably do not amount to much more than statistical noise from a small sample, in my view. Still, this certainly cuts against the notion that the polls necessarily break toward the challenger.
One debatable point about this analysis is that we have included all elections since 1968, when only some of them involve true incumbent presidents. In particular, the elections of 1968, 1988, 2000 and 2008 did not have a true incumbent president running — although Hubert H. Humphrey in 1968, George H.W. Bush in 1988 and Al Gore in 2000 were incumbent vice presidents at the time they ran for election, which may produce many of the same effects.
Still, we can rerun these numbers with those four years excluded:
This evidence is at least a little bit better for the “incumbent rule.” You still see the tendency of the incumbent candidate to overperform polls conducted in July and August. But the challenger has overperformed the polls in the other months.
However, it is not clear that this has to do with which candidate is the incumbent and which is the challenger so much as something else. In presidential years, the polls typically break toward the candidate who trails in the polls — that is, there is some regression toward the mean. A candidate who trails by 20 points in July is a favorite to do at least a little bit better than that by Election Day (although, obviously, he is a huge underdog to actually win the election).
More often than not — in an election against a true incumbent — it is the challenger who trails, so what looks like the “incumbent rule” may simply be regression toward the mean.
You can see this phenomenon fairly clearly if, for instance, you compare the margin in the September polls against the actual election results, as in the chart below. In this chart, I have included only those years in which there was a true incumbent candidate running (1972, 1976, 1980, 1984, 1992, 1996, 2004).
The regression line in the chart has a slope of less than one (specifically, it has a slope of about .7). What that means is that the candidate leading in the September polls will typically only realize about seven-tenths of his actual margin on Election Day — so, for instance, the best guess for a candidate who leads by 10 points in September is that he will win by 7 points in November.
However, the regression line also passes almost exactly through zero when it crosses the y-axis. What that means is that there has not been any bias in the polls toward either the incumbent candidate or the challenger: this has been an equal-opportunity phenomenon: A challenger who trails in the polls is likely to regain some ground — but the same is also true when an the incumbent candidate lags behind, as was the case in 1976 and 1992, when Gerald Ford and Mr. Bush rebounded late in the polls. (Although this was not the case for Mr. Carter in 1980.)
The same pattern also largely holds if you do include 1968, 1988, 2000 and 2008 in the figures.
This has been a detailed post, so perhaps an executive summary is in order:
1. The challenging candidate has typically been underrated by head-to-head polls when he is still engaged in a heated primary battle, when his name recognition is low, or both. These effects seem to evaporate by April of the election year or so, when the result of the nomination process is likely to have become clear and when the presumptive nominee is likely to have become widely known to voters.
2. The evidence is not very convincing that polls break toward the challenger otherwise. (This seems to be true both for presidential elections and for other types of elections.)
3. In presidential years, the mean-reversion tendency in polls has been much clearer than any effects from incumbent status.
4. The effects of the conventions can skew poll analysis and can lead candidates to be underrated (particularly the incumbent-party candidate after the challenger’s convention) if it is not accounted for.
5. The analysis is complicated by the fact that there are different ways to define incumbent candidates, by the small sample size, and by the fact that the way voters respond to polls may be changing. So — although I do not think the evidence for the incumbent rule is very convincing — keep an open mind to the effects of incumbency status as we gather more data from this and from future election years.