8:49 p.m. | Updated This analysis was written and published before I had seen the results of the new New York Times/CBS News poll — which had President Obama’s approval ratings dropping all the way down to 41 percent. That certainly weighs the evidence more in the direction of Mr. Obama’s numbers having declined from their February peaks — and makes November’s election look more like a toss-up than one that leans toward Mr. Obama.
But the point remains that there is a lot of noise in polling data. The New York Times and Washington Post polls showed declining numbers for Mr. Obama. But the Gallup tracking poll, also published on Monday, showed Mr. Obama’s approval rating rising to 49 percent, one of the higher figures he has posted in that survey.
This is what polling data looks like: at any given time, you are going to see some relatively stronger and weaker numbers for Mr. Obama and for the Republican candidates — both because of the different methodology employed by different polls and because of sampling error.
The same holds even in surveys published by the same polling firm; Gallup, for instance, had Mr. Obama’s approval rating at 43 percent just days ago. This is why I usually resist commenting on approval ratings data until there is a trend consistent across almost all surveys.
Timing can also be a factor: the Monday Gallup poll, for instance, was conducted entirely over the weekend, after the relatively good February jobs report had been published. It’s possible that made some difference, but who knows. In the long-run, a poll published 238 days before an election is not going to be much more accurate than one conducted 236 days before the election — unless there has been some fundamental shift in the political environment.
For now, those fundamentals look mostly the same. So far, the rebound in consumer confidence has been fairly resilient to gas-price increases, for instance.
One more thought: whatever their predictive power, polls can also affect the perceptions and the dynamics whenever there is an active campaign. For instance, I tend to see worse approval ratings for Mr. Obama as somewhat favorable for Mitt Romney in the Republican nomination battle — they make the point that the president could easily be beaten if Republicans run the right campaign, potentially underscoring his edge on electability. I’d also expect there to be be more emphasis on gas prices in news coverage — and the perception that they are probably an unfavorable factor for Mr. Obama in the near term.
—-
A new Washington Post/ABC News poll is garnering a lot of attention. It shows President Obama’s approval rating at 46 percent — down from 50 percent in a poll that the same organization conducted in February. Meanwhile, Mr. Obama’s disapproval rating increased in the poll, and his standing against the two Republican front-runners, Mitt Romney and Rick Santorum, deteriorated. The headline on the article implies that the changes have been caused by higher gas prices.
I’ll take up the gas price issue in a moment, but the first question to ask is whether the movement in the poll is real or could instead represent statistical noise.
The margin of sampling error on Post/ABC polls is plus or minus three percentage points. That means that the February poll implied that Mr. Obama’s true approval rating was somewhere between 47 percent and 53 percent, and that the March poll implied that it was somewhere between 43 percent and 49 percent.
As you can see, those ranges overlap, which certainly introduces the possibility that this is just statistical noise.
With that said, the margin of error is a somewhat arbitrary concept. By convention, it is meant to refer to cases that cover 95 percent of all possible outcomes resulting from sampling variance. Why 95 percent instead of 99 percent, or 90 percent, or 80 percent? I don’t know, actually — it’s just the industry standard. If there’s only an 85 percent chance that the decline is real rather than a 95 percent chance, do you think David Axelrod and Barack Obama would be giving each other high-fives at the White House? Probably not. The poll would still probably qualify as bad news for the president, just less definitively so.
However, we have some other evidence about Mr. Obama’s approval rating — the polls put out by other survey organizations. And that evidence points toward this survey’s reflecting statistical variance, at least in part.
As of Monday, Mr. Obama’s approval rating in the Real Clear Politics average of polls is 48 percent, and his disapproval rating is 47.5 percent. One month ago — on Feb. 12 — those numbers were 48 percent and 47.2 percent, respectively. So this average of polls has shown almost literally no change.
Or you can look at the average from the Huffington Post Pollster chart, which gives you a lot of flexibility in how you calculate the average. You can choose more or less sensitive settings, and you can choose to include or exclude certain pollsters. In the various versions in the chart below, I’ve used the different sensitivity settings and run versions with and without Gallup and Rasmussen Reports, which track Mr. Obama’s approval rating daily and therefore tend to dominate the trend line under this particular method.
But the story is basically the same. The most aggressive settings show Mr. Obama’s approval rating having declined by about one point. The more conservative settings instead show it increasing some. There just hasn’t been a lot of difference.
My gut feeling on how to read this data is more in line with the sensitive settings — that it has probably declined by one point or so from a month ago, but probably not much more than that, and that it is quite a bit higher than it was two or three months ago.
So suppose that Mr. Obama’s approval rating has declined by one point — or even by four points, as under the Post/ABC poll. Can we attribute the change to gas price increases?
No, probably not. The poll showed that many Americans are unhappy about gas prices. Would you expect otherwise when gas is nearly $4 per gallon in many places?
But there’s not really a good way to tell what effect this has had on Mr. Obama’s approval rating. Among other things going on during this period, we’ve had a big fight over contraception, a hotly contested R
epublican nomination race, and jobs reports that have generally been pretty good but suggest that the economy still has a way to go.
Every now and then, the timing of a news event is precise enough that we probably can attribute a change in approval ratings to it. There was very little doubt, for instance, that the killing of Osama bin Laden almost immediately improved Mr. Obama’s approval ratings, while the contentious resolution to the debt-ceiling debate lowered them.
Other than perhaps a stock market crash or a major economic report that massively defies expectations — last month’s jobs report might have been the exception that proved the rule — most economic factors evolve more gradually than that. And they don’t necessarily work in tandem with one another. It’s certainly possible, for instance, that the improved jobs picture is pushing Mr. Obama’s approval ratings upward, while higher gas prices are constraining them — leading them to remain roughly stable overall.
It’s also possible that gas prices aren’t having very much of an effect. The empirical evidence suggests that gas prices play a more indirect than direct role in determining a president’s fortunes. They matter to the extent to which they can affect the broader economy, and they certainly can do that — see this forecast from Goldman Sachs’s Jan Hatzius, for instance. And gas prices have historically been fairly strongly correlated with consumer confidence.
But there just isn’t a lot of evidence that they have a strong effect on a president’s approval rating independent of the broader economy. President Bush and John McCain did not seem to benefit when gas prices plummeted in September and October 2008, since the reason for that was the total collapse of the financial system. Consumers weren’t so happy about paying only $2 a gallon for gas when they had lost $2,000 in their pensions — and maybe were about to lose their jobs.
I qualify this because it is incredibly difficult to untangle cause and effect when you have a series of highly correlated economic indicators. This is one reason that economic forecasts are notoriously bad — seemingly foolproof predictions based on so-called leading indicators often fail miserably in the real world. To the extent that political predictions depend on economic predictions, this creates even more uncertainty, since even more variables and unknowns are introduced into the problem.
I encourage people to take a more holistic view of the economy rather than making a fetish of one or two particular indicators — many of the major economic indicators, like gross domestic product, income growth and jobs growth, have been about equally strongly correlated with presidential re-election rates over the long run. And none of them are perfect.
The bottom line is that we’re at a point where Mr. Obama can make a credible case that the economy is improving under his stewardship — and Republicans can make a credible case that it’s too little and too late. Mr. Obama probably has a very slight edge if you’re forced to make a bet — but a “snap election” called for five weeks from today would be very close, with the outcome highly uncertain.
The election might not be quite as close if we continue producing 225,000 jobs per month — not just because the jobs numbers are important unto themselves but because that sort of steady improvement would be a sign that the economy had enough favorable momentum to overcome most of its headwinds. But energy prices could constrain that growth. Or, especially in the event of an oil price shock from developments in the Middle East, it could reverse it, giving Republicans the stronger side of the economic argument again.
This is the same old story, in other words. In overreacting to every tick in Mr. Obama’s approval ratings, you’ll risk losing sight of the bigger picture.