Skip to main content
ABC News
When You Assume, You Make a Mess Out of Your Poll

It’s a bit difficult to reconcile the results of three questions from the new Washington Post/ABC News poll on health care reform:

16. Overall, given what you know about them, would you say you support or oppose the proposed changes to the health care system being developed by (Congress) and (the Obama administration)?

Support 46%, Oppose 48%

22. Would you support or oppose having the government create a new health insurance plan to compete with private health insurance plans? Do you feel that way strongly or somewhat?

Support 55%, Oppose 42%

23. Say health care reform does NOT include the option of a government-sponsored health plan – in that case would you support or oppose the rest of the proposed changes to the health care system being developed by (Congress) and (the Obama administration)?

Support 50%, Oppose 42%

This is the Washington Post’s attempt at interpreting these results:

But it is the public option that has become the major point of contention, with support for the government creation of an insurance plan that would compete with private insurers stabilizing in the survey after dipping last month. Now, 55 percent say they like the idea, but the notion continues to attract intense objection: If that single provision were removed, opposition to the overall package drops by six percentage points, according to the poll.

Without the public option, 50 percent back the rest of the proposed changes; a still sizable 42 percent are opposed. Independents divide 45-45 on a package without the government-sponsored insurance option, while they are largely negative on the entire set of proposals (40 percent support and 52 percent oppose). Republican opposition also fades 20 points under this scenario.

The decision to back away from the provision might hurt Obama among his base, but not dramatically so, as 88 percent of liberal Democrats support the reform plan as is, 81 percent without the public option.

This, I suppose, is the Occam’s Razor interpretation. Even though a majority of people support the public option, its inclusion is a “deal breaker” for a decent fraction of the opposition — enough that support for the plan increases overall if it is removed, even though some people (about 6 percent of “liberal Democrats”) will cease to support the plan under those circumstances.

However, this is not the only interpretation — and it may not be the correct one. One problem is that both the poll and the article assume that the public assumes that the public option is the default condition of the health care reform plan. But as anyone who is following the health care debate knows, this is hardly a safe assumption. Some versions of the “proposed changes to the health care system being developed by Congress and the Obama administration” — like the one which was passed by the House tri-committee — include a public option. Other versions — like the draft prepared by Max Baucus’s Senate Finance Committee — do not. And the President himself is on the fence. As a matter of semantics at the very least, it is not really proper to state that “If [the public option] were removed, opposition to the overall package drops by six percentage points,” because it is not clear that the public option is in the package in the first place.

So what? — I can hear you saying. At first the Washington Post asks a question about health care reform without including any specificity about the public option. I’ll take you at your word, Nate, that some people will start out by assuming that the plan in fact has a public option and others will not. But once you specify that the plan does not have a public option, support rises. Does that not nevertheless prove that punting the public option would improve the plan’s popularity?

No, it doesn’t. It’s suggestive of that result, certainly. But it’s a pretty far cry removed from proof of it. Let me explain why.

When you’re specifying that the plan does not include a public option, you’re really doing two things. Number one, you’re taking the public option off the table. But number two, you’re providing specificity. And what the health care polling has consistently shown over the past few months is that the more specificity you provide, the more support for the package rises.

I can buy that specifying that the plan would not include a public option would improve support for it. But it might also be the case that that specifying that the plan would include a public option would also improve support for it. The respondents may be reacting to the specificity more than anything having to do with the public option itself.

What the Washington Post should have done is the following: break their sample into two halves. To one half, ask again about the health care package, but specify that it will not include a public option. To the other half, ask again about the health care package, but specify that it will include a public option. Then compare and contrast. If the Washington Post had done this, it would not surprise me if support for the plan increased among both subsamples.

There is another problem too, which is that the second question on the public option — question #23 — omits a key word from the original question. The word in question is “insurance”. Whereas the first question correctly stipulates that the public option is something having to do with government-provided insurance — something which most polls suggest is popular — the follow-up comes closer to implying that the government would become involved in the provision of health care itself, something which is probably not very popular. Since question #23 immediately proceeds question #22, the Washington Post probably assumes that people will retain the necessary context: the public option is an insurance thing, not a “government takeover” thing. But some people will not assume this — because some people are very confused about health care reform. We know, moreover, that polling on the public option is extremely sensitive to seemingly small changes in wording. There are probably not a lot of people who will be confused by all of this — but I assure you, there are some.

Basically, the Washington Post is assuming too much of its respondents — making assumptions which might seem obvious in the Washington Post newsroom, but which might not be so apparent to the 1,007 random Americans that they polled.

From my vantage point, what the poll “proves”, if anything, is that specificity will be helpful to the Democrats. They should either insist on the public option or remove it — but keeping their options open may be doing little more than confusing the public. And the pollsters.

Nate Silver is the founder and editor in chief of FiveThirtyEight.