The polls suggest a tight race in Alabama and Mississippi, which will hold their primaries on Tuesday. Our polling-based forecasts show Newt Gingrich with a very small lead in Alabama and Mitt Romney with an equally small one in Mississippi. Rick Santorum seems to have fallen to third in both states, but remains in striking distance.
I’ve been placing less emphasis on the polls than usual, however. The reason is that polls in these states have a pretty awful track record.
If you had looked at the Real Clear Politics average of polls in Alabama in advance of the 2008 primaries, for instance, you would have called the Republican and the Democratic races wrong. Hillary Rodham Clinton had a nominal advantage in the Democratic race there, leading in the Real Clear Politics average by about one percentage point. But in fact, Barack Obama won. It wasn’t even close; he carried the state by 14 percentage points.
A similar problem was evident on the Republican side. The polls were wildly divergent from one another — showing everything from a 16-point lead for Senator John McCain to a 9-point advantage for Mike Huckabee. But the average put Mr. McCain about four percentage points up in Alabama. Mr. Huckabee won the state by about four percentage points instead, however.
Alabama and Mississippi haven’t played an important role in primaries that often, but in one important prior case — the Democratic race in 1984 — the polls were also pretty bad there. Surveys substantially overestimated Walter Mondale’s standing — one survey had him with as much as 48 percent of the vote, but he got just 35 percent. They had Gary Hart finishing about 10 percentage points ahead of Senator John Glenn for second place, when the two candidates about tied for second. And Jesse Jackson did substantially better than was forecast for him by the polls.
There are different ways to calculate the error in a poll, but the technique we’ll apply here is to look at how much the survey missed on a candidate-by-candidate basis after adjusting for the number of undecideds that the poll contained. For instance, if a poll projected Mr. Mondale to get 45 percent of the vote after removing undecideds from the total, but he actually got 35 percent, that would be a miss of 10 percentage points.
On average, among the primary polls in our database from 1972 to 2008, surveys in Alabama and Mississippi conducted in the final two weeks of the campaign have missed by an average of 5.4 percentage points per candidate. That’s considerably larger than the average for primaries in all states, which have missed by an average of 3.5 percentage points by the same measure.
The problems also seem apparent in the other states that make up the Deep South — Georgia, Louisiana and South Carolina. Polls in those three states have missed by an average of 4.2 percentage points — or 4.4 percentage points if you also include Alabama and Mississippi in the total.
There do not seem to be as many issues with polls in border states in the South, nor in Florida or in Texas, which have some commonalities with the region but also have their own distinct political characters.
The sample size is not all that large, however, so whether these trends qualify as statistically significant depends on how it is calculated. If each poll is treated as independent from the next one, there has been a statistically significant tendency toward greater polling error both in the Deep South as a whole, and an additional effect for Alabama and Mississippi in particular.
If instead all the polls in the same election cycle are treated as being related to one another, however, the effective sample size declines, and the effect falls just short of statistical significance.
Still, this is something to be mindful of, and I think there might be a plausible hypothesis to explain it. It’s a little speculative, but bear with me for a moment.
Being polled is in some ways a strange thing — you are receiving an unsolicited call from a stranger, and being asked to reveal private information in the form of your voting preferences.
Polls can sometimes have problems because of social desirability bias — the tendency to provide an answer that you think might seem most acceptable to the stranger on the other end of the line, rather than what you really think.
This bias is potentially stronger in cultures that have stronger codes of etiquette, and where people are more self-conscious of the front they present to strangers. This is pertinent in some Asian and Asian-American cultures, for instance. Polls of Hawaii, where there are many Japanese-Americans, have a bad track record; one survey there somewhat infamously predicted a win for George W. Bush in 2004, but John Kerry instead took the state by 9 percentage points.
Etiquette also remains more intact in the South, and especially in the Deep South, than in most other parts of the country. If so, polls there could encounter similar problems.
But whom is this likely to favor? That is more uncertain. There has been some tendency for polls in the Deep South to understate the standing of Southern candidates, but it is not statistically significant.
If the causal mechanism is social desirability bias, however, it might be that the polls exaggerate the standing of candidates who are seen as more broadly acceptable and as less offensive, or who have the support of the establishment. In this election, that might be Mr. Romney, who won the endorsement of Gov. Phil Bryant of Mississippi.
It could also be the case that polls conducted by automated script have fewer problems than those that use live interviewers — there’s no need to be polite to a computer. We haven’t really seen consistent differences in levels of candidate support based on the mode of conducting the poll in Alabama and Mississippi so far this year, however.
And polls conducted by automated script — as well as some that use traditional methodology — have another problem in the Deep South: they don’t call cellphones. The incidence of cellphone-only households is especially high in the South, particularly in Mississippi, where about 35 percent of the adult population had no access to a landline as of 2010. These voters just aren’t being included in most of these surveys. Polling in primaries and caucuses is often an adventure, b
ut this holds especially for Tuesday’s contests.