On Sunday, voters in the Crimea region of Ukraine overwhelmingly chose to secede and become a part of Russia. Crimean officials said nearly 97 percent of voters backed the move, casting ballots as the peninsula remained occupied by thousands of Russian troops.
The huge margin lent credence to U.S. government claims that the vote was tainted by “intimidation from a Russian military intervention”; the high turnout rate and local media reports suggested fraud occurred. That secession won came as little surprise, though — except, perhaps, to readers of polls conducted by the Kiev International Institute of Sociology, which has worked in the region since 1990 and has a strong reputation with U.S. pollsters who work globally.
Forty-one percent of Crimeans in the latest KIIS poll, conducted from Feb. 8-18, said Ukraine and Russia should merge into one state. That percentage has ebbed and flowed in recent years, in part because of the small sample size of any one of Ukraine’s 24 oblasts (provinces) in a national poll. Based on recent years, 41 percent could be an overestimate — just one-third of Crimeans wanted Ukraine to join Russia last year, and fewer than 1 in 4 did in 2012.
Yet when I talked with Volodymyr Paniotto, general director of KIIS, he didn’t think the polling history meant much on the eve of the referendum, for several reasons. Russian occupation made face-to-face polling in Crimea difficult this month, so there were few reliable gauges of whether Russia’s aggression had shifted opinion. Paniotto also hesitated to ask about the referendum because doing so could seem to legitimate it, a position at odds with Ukraine’s government. And he said Russian media flooded Crimea with propaganda that may have swayed opinion. “My Russian friends said they don’t remember, even in Soviet times, such intensive propaganda as on mass media in the last month,” Paniotto said. And perhaps many Crimeans wanted just their region, rather than all of Ukraine, to merge with Russia.1
Most crucially, the referendum worded the question differently than Paniotto and most other pollsters would have — at least if they were seeking honest responses. Crimeans chose between two options, and neither was the status quo: One was to join Russia, and the other was to return to the constitution of 1992, which leaves it to Crimea’s legislature to decide whether to leave Ukraine. That body’s members already had indicated they wanted to do so before the vote, and declared Crimea’ independence from Ukraine after the vote. (Each ballot also led with the reunification option, which studies have shown can inflate vote totals.)
“The questions are very bad,” Paniotto said on Thursday, ahead of the vote. He has studied how wording has influenced Ukraine-wide referenda by swaying voters without strong opinions. He reckons one-third of voters in Crimea fit that description. “What question we ask,” he said, determines “what we receive.”
The referendum was “just theatrical show,” Paniotto said early Monday, after the results were in.
The best measure of Crimeans’ opinion on secession may be polls such as his, rather than the referendum itself. The people who conduct and study surveys internationally say the best public-opinion data from the former Soviet republic is as reliable as any around the world.
“The survey data you get in Ukraine, even on the most sensitive issues, is very reliable data,” said Rakesh Sharma, director of applied research for the International Foundation for Electoral Systems, which has been polling in Ukraine, often with KIIS, since 1994. Its work is funded by the U.S. Agency for International Development.
One big advantage for pollsters: working in the country, with a per-capita GDP below Iraq’s, is cheap, costing less than one-fifth as much to add a question to a Ukrainian poll as it does for a U.S. survey with the same number of respondents, according to Steven Kull, director of WorldPublicOpinion.org. “There’s a lot of worthwhile polling that could be done right now,” Kull said. “I’m tempted to jump into it.”
When the Soviet Union dissolved in 1991 and Ukraine gained independence, pollsters found a country full of people eager to tell anyone, including strangers, their opinions about their country and its governance — an ideal scenario for survey researchers.
The “attitude of the population was very, very positive,” Paniotto recalled. He had to work to build polling infrastructure and train interviewers in a country with little independent polling during the Soviet era, and with two major languages (Ukrainian and Russian). Once his staff went into the field, respondents often invited them in for tea; they had to learn how to leave politely and move on to the next home.
“After the Soviet time, when no one was interested in your opinion, then a special person came to your house and asked your opinion,” Paniotto said. “We had a response rate of about 90 percent.”
Pollsters proved their mettle before the last Ukrainian presidential election, in 2010, by successfully forecasting Viktor Yanukovych’s victory in the first and second rounds of voting.
Just six years ago, Crimea wasn’t the most pro-Russia region of Ukraine. At the time, as now, about 40 percent of Crimeans supported joining Russia — but support for the idea was about as high in five other oblasts and far higher, above 50 percent, in Luhansk, on the eastern border with Russia, according to KIIS data. Since then, Ukrainians in all of those regions except Crimea and the eastern oblast of Donetsk have moved sharply away from the Russia merger option.
Those polls were conducted among respondents less enthusiastic than their tea-serving predecessors just after independence. Andrey Milekhin, president of Romir Holding, which bills itself as the region’s largest independent polling company and polls for U.K.-based ORB International, detects “severe weariness” in the population. He said residents are jaded by frequent elections and what he called “discredited” polls, commissioned by parties to produce favorable results.2
“Ukraine has very good survey capacity, and very good surveys are done, but there also are some bad surveys being done, around election time,” Sharma said.
Among evidence that Ukrainians are weary of polls is a decline in response rates — down to roughly 65 percent nationally, according to Paniotto, and down to 50 percent in cities. James Bell, director of international survey research for the Pew Research Center, said its surveys’ response rates have fallen to 19 percent in Ukraine — higher than Poland’s 12 percent but well below 82 percent in Kenya and 79 percent in Indonesia. That makes results less reliable as a proxy for overall public opinion than they used to be, particularly for men and young people, who respond at lower rates than everyone else.
Locked security gates more often thwart pollsters these days, but when they do get through, “it’s surprisingly refreshing how open and cooperative people are to talk,” Bell said. Those who open the door are willing to spend 35 to 45 minutes telling strangers how they feel, he said.
Before police clashed with protesters last month, the since-ousted Yanukovych’s administration mostly tolerated dissent, Sharma said. “There is a tradition in Ukrainian discourse that citizens are allowed to voice their opinions. It’s one of the reasons why reaction against a government clampdown was particularly strong in Ukrainian public opinion, going against the grain of what Ukrainians expect to be able to do.”
Paniotto isn’t sure whether he’ll keep polling in Crimea after secession. “Now nobody is psychologically ready to accept that Crimea will be Russian,” he said, “and we have not yet discussed any scenarios for such situations.”