Sorry, Jack Welch. It doesn’t look like President Obama’s henchmen mucked with the unemployment rate after all. Or at least that’s the conclusion of an investigation by the Census Bureau’s inspector general.
Readers may recall that back in October 2012 — a month before the presidential election — Welch, the former General Electric chairman, took to Twitter to allege, or at least strongly imply, that allies of Obama were manipulating jobs data to make the economy look better than it really was.
Shortly after the Bureau of Labor Statistics reported that the unemployment rate had fallen to its lowest level in nearly three years, Welch tweeted:
Unbelievable jobs numbers..these Chicago guys will do anything..can't debate so change numbers—
Jack Welch (@jack_welch) October 05, 2012
Welch’s comments were roundly mocked at the time by financial journalists and others. But then last year, New York Post business columnist John Crudele joined the fray with a startling allegation: Citing a “reliable source,” Crudele said government workers had cooked the numbers in the months leading up to the election. Specifically he said that census workers in the Philadelphia region had falsified interviews that go into the survey that forms the basis of the monthly unemployment figures.
Crudele’s claims were much more specific than Welch’s, which made them harder to dismiss (although most mainstream media were still skeptical). Members of Congress demanded an investigation, and they got one: The Census Bureau’s Office of Inspector General looked into the issue.
Six months later, the results are in: The investigation “did not find any evidence to support allegations that supervisors in the Philadelphia Regional Office manipulated, or attempted to manipulate, the unemployment rate prior to the 2012 presidential election,” the report concludes. (The report does recommend a few procedural changes to make the data more reliable, however.)
Much of the report focuses on the details of the fraud accusations and why they’re unfounded. (Among the more interesting tidbits: Data collected by the implicated worker actually tended to push the unemployment rate up, not down. And in any case, he was fired more than a year before the controversial pre-election report.) More interesting to DataLab readers, however, may be the report’s detailed explanation for how the monthly jobs numbers are calculated — and how hard they would be to fake.
A quick refresher: The unemployment rate is based on the Current Population Survey, a monthly survey of about 60,000 households. (The payroll figures — the “nonfarm jobs” added or lost each month — are based on an entirely different survey.) The Census Bureau conducts the survey, then sends the data to the Bureau of Labor Statistics, which does the math and publishes the results.
All of the survey results, down to the individual interview records, are released to the public, which means falsifying the report isn’t as simple as just tweaking a couple numbers. Someone would have to alter (or make up) the individual interviews. That would require a pretty massive conspiracy: According to the Census Bureau’s calculations, reducing the unemployment rate by three-tenths of a percentage point (the amount of the October 2012 decline) would require 78 census surveyors to work together to switch everyone they interviewed from “unemployed” to “employed.” That kind of fraud would almost certainly be detected by higher-ups at the Census Bureau, which means they would need to be in on the conspiracy, too.
But perhaps the best evidence against a conspiracy theory is this: For all of Welch’s skepticism, that October jobs report didn’t turn out to be that much of an outlier. The unemployment rate has continued to fall in the months since the election, and a wide range of other indicators — many of them private-sector sources that are presumably outside the Chicago guys’ control — have shown the same trend of gradual improvement.
Still, with the next employment report set to be released Friday, the IG’s investigation provides a timely reminder to treat all jobs numbers with caution. Not because they’re politically motivated, but because of the uncertainty inherent in all survey-based data. Here at FiveThirtyEight, we often remind readers about sampling error, the risk the people interviewed in a survey aren’t perfectly representative of the population as a whole. (The margin of error for Friday’s headline payroll figure is about 95,000 at the 90 percent confidence interval, as Neil Irwin noted Thursday.) But sampling error is just one kind of error. Survey respondents can lie, misunderstand a question or forget the answer. Pollsters can enter responses incorrectly, misread questions or, yes, falsify interviews entirely.
So don’t mistrust Friday’s numbers because they’re fake. Mistrust them because, like all survey data, they’re approximations.