President Trump’s executive order banning travelers from seven predominantly Muslim countries from entering the U.S. has been perhaps the most controversial act of his presidency so far. The order set off a firestorm of protests and numerous court challenges. But it’s still difficult to measure how many Americans support the policy. It’s hard to distill complex policy into a short poll question, so different pollsters wound up using different wording. Additionally, pollsters conduct their surveys using a variety of methods and during different time periods. The result was that some polls found a plurality in favor of the executive order while others found a plurality against it.
With the Trump administration reportedly close to issuing a revised version of the executive order, it’s worth looking at a new study from Morning Consult, which was shared with FiveThirtyEight, that tested how question wording affected public support for the travel ban. Morning Consult asked 96 versions of the travel ban question to a total of more than 22,000 people in surveys that ran from Feb. 10 to Feb. 27. The results indicate that approval for the executive order remains mostly stable over time and across alternative wordings. This suggests that survey house effects or mode effects (whether the poll was conducted with live interviewers, online, etc.) explain much of the varying levels of support for the executive order. (More on this in a moment.)
First, here are some of the key Morning Consult findings in more detail:
1. It makes no difference whether the people kept out are described as being from “Muslim” countries or countries “linked to terrorism.”
A lot of the debate about Trump’s executive order was over whether it as a de facto “Muslim ban,” or whether it was, as the administration argued, a common-sense precaution to keep potential terrorists out of the U.S. But Americans didn’t see much of a difference between the two framings. A slight majority, 53 percent, were in favor of a ban on travelers from “seven majority-Muslim countries.” And an almost identical 54 percent supported a ban on immigrants from “seven countries linked to terrorism.” Similarly, 55 percent of voters favored a ban on travelers from “seven countries linked to political instability and violence.”
2. Americans did seem to care whether permanent residents and visa holders were kept out.
Trump’s new executive order will reportedly exempt green card holders (who are legal permanent residents) and visa holders, and this could make a difference politically. When voters were asked about keeping “people” out of the country, approval for the ban was 55 percent. When voters were asked about keeping “people, including U.S. lawful permanent residents and visa holders” out of the country, approval dropped to 51 percent. That’s not a huge difference, but it’s statistically significant. Indeed, of the 10 versions of the question that garnered the most support, only one mentioned permanent residents and visa holders. Additionally, some of the most vivid news reports regarding the original order were about green card holders being kept out of the U.S., unable to return home. If the second order is implemented without generating stories like that, the public might find it easier to accept.
3. Support doesn’t appear to depend on how long the ban lasts.
When Trump’s executive order first came out, it left the door open for the possibility of extending the temporary ban. I wrote at the time that “the longer Trump’s immigration restrictions remain in place, the more pushback he might encounter.” That was based on previous polls and may still may prove right, but, at least in terms of question wording, how long the ban lasts doesn’t seem to affect its support. Whether the order is described as temporary or as 90-days long, or whether no time limit is mentioned, approval remained at about 53 percent or 54 percent.
4. The timing of the poll didn’t matter very much.
As courts ruled against Trump and coverage of the protests against the executive order dominated the news, one might have expected opinion to turn against the travel ban. But support for the executive order has remained stable. When Morning Consult first polled about the order, from Feb. 2 through Feb. 4, approval came in at 55 percent. Over the course of this 22,000-voter survey, which ran from Feb. 10 through Feb. 27, support was 54 percent.
This matches the trend in Trump’s job-approval rating, which has remained fairly steady over the course of his presidency so far.1 As with the president himself, Americans either like his executive order or they don’t. The timing and wording of poll questions made little difference.
So we’re still left with this question: Why did some polls find that more people approve of the order than disapprove and other polls find the opposite? There’s a simple explanation: Polling is inexact. All polls have a margin of error, so we shouldn’t be surprised when surveys don’t find the exact the same thing.
Another possibility is that polls conducted via different technologies are producing different results — this is the mode effect I mentioned earlier. Morning Consult, which conducts its surveys online, has tended to find more support for the executive order than other pollsters. A study by Ariel Edwards-Levy and Grace Sparks at the Huffington Post showed that the polls that found the most support for the ban were usually done online or by automated phone call. On an issue where many respondents — regardless of their personal stance — may believe that there is is a politically correct answer (to be against the executive order), it’s possible that some people are lying to live interviewers. Of course, online and automated phone surveys can miss certain segments of the population that live interview surveys tend to reach. For that reason, live interviewer surveys have, on average, been more accurate over time.
We’ll probably never know the true level of support for Trump’s executive order — unlike in an election, there’s no vote to measure the polls against. The best we can do, as usual, is average the different polls together and hope that brings us closer to the truth.