Most Iowa polls showed Donald Trump winning the state’s Republican caucuses. He didn’t. Some Iowa polls showed Hillary Clinton winning Iowa easily. She didn’t. It’s notoriously hardto poll Iowa, but what can pollsters learn from Monday night’s results to improve their work over the next few months — and for the 2020 caucuses?
One of the biggest lessons was a simple one: Keep on contacting voters as late as possible.
“Stay in the field until the night before the event, if possible,” said Christine Matthews of the Republican consulting firm Burning Glass Consulting. Matthews was one of eight pollsters who were active in Iowa and who responded to a small poll of pollsters we conducted the day after the caucuses to ask how they thought the polls did, and what lessons they learned. (You can find the questionnaire in this PDF, all the responses on GitHub, and a list of the pollsters who responded in the footnotes.1)2
“Look for the trend at the end,” said Doug Kaplan, managing partner of Gravis Marketing.
That sounds like a wise approach because of the success by two pollsters whose final Iowa polls started at a later date than anyone else: last Friday, January 29. The two pollsters, Emerson College and Opinion Savvy, both showed Donald Trump leading Ted Cruz by just one percentage point, with Marco Rubio either close behind or tied. Most other polls started before the last debate and many finished before it, too. They generally showed Trump with a bigger lead and Rubio trailing him by more than 10 percentage points. Cruz finished with 28 percent of the vote, Trump with 24 percent and Rubio with 23 percent. (In our poll, pollsters from both Emerson and Opinion Savvy named their own polling organization as among those who did the best job in Iowa.)
Then again, polling late, after the last debate, is difficult. “I guess if there’s a lesson, it’s to keep on polling through the night before the election,” said Tom Jensen, director of the Democratic firm Public Policy Polling. “But polling’s never been more expensive so I don’t know how many organizations really have the resources to do more than they already are.”
New Hampshire poses a specific, daunting problem for anyone who wants to poll late. The next Republican debate is on Saturday night, and one of the two days between the debate and next Tuesday’s primary in the state is Super Bowl Sunday. “So it’s basically impossible to poll in New Hampshire after the debates at a time when you will reasonably be able to reach people,” Jensen said. “We are leaning toward skipping polling New Hampshire for that reason.”
Also, in the Democratic caucus, polling late wasn’t as much of a clear advantage. Emerson’s late poll showed Hillary Clinton leading Bernie Sanders by 8 percentage points. The two finished in a virtual tie, which was closer to the results found by other pollsters who’d started earlier than Emerson did. So another lesson from Iowa ought to be one that usually applies when drawing conclusions about quadrennial events: Don’t extrapolate too much from a small number of events.
While many polls greatly overestimated Trump’s support, several polling watchers — including my boss — emphasized that the polls didn’t have such a bad night.
Most pollsters who responded to us seemed glad it was over. Just one of eight said polls had a good night, three said they didn’t and four were unsure.
“It wasn’t a total disaster, but most of the polls failed to capture some of the basic trends,” according to one of the three “no” votes, Matthew Towery, founder of Opinion Savvy. “I don’t know if it was simply that some were in the field too early, or if their screening, turnout, and weighting models were just off. Nevertheless, this will likely go down in history as the worst round of modern polling for the Iowa caucus to date.”
Two pollsters said Selzer & Co. did the worst job in capturing the horse-race numbers. Clare Malone’s profile for this site last week called the company’s founder and president Ann Selzer “the best pollster in politics,” but Selzer hasn’t been immune to the challenges of polling Iowa. For the third straight Republican presidential caucus, her final Iowa poll for The Des Moines Register underestimated the support of the eventual winner, who each time was the favorite choice of evangelical voters. “They continue to undercount the evangelicals in the GOP side,” said Spencer Kimball, advisor to the Emerson College Polling Society. But Jensen defended Selzer’s work: “Her poll was probably correct at the time she did it. It’s unreasonable to think that everything is going to stay in amber as soon as her field period ends.” (Another pollster named Selzer as the pollster who did the best job for catching general trends and coming close on the levels of support of candidates below the top three.)
Selzer was one of 18 Iowa pollsters who didn’t respond to our poll, but she did respond to an email about her work. “We modeled higher evangelical turnout,” Selzer said. “Who would have expected 64%?3 Especially with that community, things happen the Sunday before the caucus that end up in no one’s data. I’m feeling good today. Caucuses are hard to poll — that’s no secret to pollsters or my clients.”
We also asked pollsters for their predictions for New Hampshire and beyond. Though FiveThirtyEight’s polls-only forecast projects Trump to win New Hampshire by nearly 20 percentage points as of Tuesday, no one thought Trump would win by such a big margin. Two pollsters said he’d win by between 10 and 19 points, four said he’d win by a single-digit margin and one said he’d finish second for the second straight contest. Just one of seven pollsters predicted Trump would be the Republican nominee; the rest picked Rubio.
All seven pollsters who predicted the Democratic nominee, meanwhile, said it would be Clinton.
CORRECTION (Feb. 2, 10:40 p.m.): An earlier version of this article incorrectly described the opinion poll forecasts for Donald Trump before the Iowa caucuses. The polls largely overestimated his support, not underestimated it.
Hey, you should sign up for our weekly 2016 newsletter, “What I Thought About Over The Weekend.”
Listen to the latest episode of the FiveThirtyEight politics podcast.