Skip to main content
ABC News
How To Make Sense Of Conflicting, Confusing And Misleading Crime Statistics

Crime statistics often are confusing, misleading and incomplete — and rarely more so than at the start of a new year, when cities start reporting last year’s crime totals. Numbers out this week show a surge in homicides in many cities, adding urgency to the usual early-January headlines, but you should view them with extreme skepticism.

The New York City Police Department, for example, has been trumpeting what it calls record-low crime rates in 2015 — in the commissioner’s media appearances, in a press release and in tweets — without mentioning that homicides, rapes and robberies all rose from a year earlier. Meanwhile, the police chief and his predecessor are embroiled in an ongoing spat over the reliability of the department’s reporting of murder and shooting stats.

Late last month, former NYPD commissioner Raymond Kelly said current commissioner William Bratton tried to deflate the city’s shooting totals by directing officers not to count people injured by broken glass caused by gunfire, or whose clothes but not bodies are hit in shootings. Bratton says such incidents have been omitted consistently since the department started tracking shootings during his first stint as commissioner in 1994. But he pointed out that both types of shootings are counted as aggravated assaults in official stats reported to the FBI.1

The conflicting messages make New York a good case study in the limits of crime reporting. Here’s a primer for how to understand crime numbers, in New York and beyond, and what they say and don’t about how safe we are from crime.

Official crime reporting is slow. Headlines are based on cities’ unofficial stats. Only some of those get reported to the Federal Bureau of Investigation’s Uniform Crime Reporting system, the official national repository of crime data. Much of the detailed crime information some cities collect doesn’t go into the UCR. And the FBI won’t release the 2015 numbers for the more than 18,000 departments that report to it until the fall.

Some cities report their own crime data in closer to real time, but it varies widely. I contacted dozens of police departments over the past week to ask for their year-end homicide totals. Some responded promptly. Others didn’t respond, or said they’d treat the request like a public-records inquiry that would take up to 10 days, or said they wouldn’t have their year-end count until March.

Just last week, the NYPD expanded the data it was sharing outside the confines of UCR. The department published data on criminal incidents that Bratton called part of a “continued effort to promote transparency,” yet it still leaves the country’s biggest city well behind much smaller peers in the level of detail it shares about crimes.

There are lots of official crime numbers, and they don’t all move together. New York isn’t the only place this year where crime trends look different depending on which crime indicators you’re studying. In Chicago and Oakland, homicides rose but overall crime fell. That’s not unusual — the Brennan Center found that crime overall was down in the country’s largest cities in 20152 even as homicides rose. (Los Angeles and Charlotte, North Carolina, were clear exceptions with crime up around 10 percent.) And something similar happened in the last two years, too. In 2014, the difference among the country’s 60 most populous cities’ homicide rates3 explained just 60 percent of the difference4 in their rates of violent crimes and overall crime — which also includes property crimes. Homicides tracked even less with broader crime indices in 2013, explaining just 53 percent of the variation.

That means that while, in general, cities with higher homicide rates also have higher violent-crime rates and property-crime rates, there are plenty of exceptions — leaving opportunities for mayors and police chiefs to cherry-pick the most favorable numbers to tout.

On the other hand, there are valid reasons to report violent crime separately from overall crime. Otherwise, several different types of crime would be lumped together — treating one homicide as equivalent to one car theft.

Crime rates can vary a lot, not just among cities, but within cities, too. On his blog I Quant NY, Ben Wellington dug into the new release of NYPD data — covering six felony categories from January through September 2015 — and ranked 188 neighborhoods by crime rates. But he found the rank would differ, sometimes significantly, if he looked only at murder, or only at violent crime. A neighborhood’s rank for murder correlated fairly well with assault and robbery but much less so with property crimes.5

Your risk of falling victim to a crime depends not only on where you live in a city, but on other factors, including whom you spend time with. Some police officials in cities with higher homicide totals in 2015 — including Washington, D.C., Portland, Oregon, and Milwaukee — emphasized that many homicide victims and perpetrators were involved in gangs, or had criminal records.

What’s undoubtedly true is that people in inner cities — particularly young black men — suffer a disproportionately large share of homicides.

Many crimes aren’t defined in a uniform way across cities. Reporting often relies on the subjective assessment of police officers or that of their supervisors. Counts of shootings like the ones at the center of the New York City dispute aren’t even official FBI stats. The FBI handbook telling police departments how to define crimes doesn’t contain the word “shooting.” Official stats generally include fatal shootings with other killings under homicides, and combine nonfatal shootings with aggravated assaults committed without guns.

Most police departments don’t keep separate counts of shootings. “You would have to pull every report and read it and determine if it fits within the criteria you are trying to break it down into,” said Melissa J. Bujeda, a spokeswoman for the Jacksonville, Florida, Sheriff’s Office. New Orleans, Baltimore, Indianapolis, Cleveland, Cincinnati and Seattle are among the exceptions that do count shootings. Oklahoma City keeps counts specifically of drive-by shootings — there were 58 incidents last year, 55 the year before and 192 in 2012, the highest number since the department started counting in 2007, according to a police department spokeswoman.

The lack of standardization leads shooting counts open to disputes like the one in New York. “That’s why I prefer crimes defined and classified according to UCR rules,” Richard Rosenfeld, a criminologist at the University of Missouri-St. Louis, said in an email. “I have much greater confidence in the reliability of aggravated assaults committed with a firearm than I do in ‘shootings.’ ” (Unfortunately, the FBI doesn’t separate assaults by firearm from other assaults for individual cities in the data it reports.)

For what it’s worth, homicides are up — though probably by less than what you’ve read. I compiled data for the 60 most populous cities in September and found that homicides had in fact risen by 16 percent — a significant increase, but less than many headlines suggested and far below the highs reached in the 1990s. Since September, the rise in homicides has grown in profile — mentioned by the FBI director and by a Republican presidential candidate in a nationally televised debate — but not in magnitude. I updated my earlier analysis, using year-end counts when possible,6 and found that homicides remained 16 percent higher than in 2014 in the biggest cities.

If it can seem from media coverage like crime is on the rise, that’s because rising crime gets covered. In the cities for which I found the latest data reported in a news article,7 homicide rates were up on average by 22 percent. In other cities, where homicide didn’t make headlines, rates were up by 14 percent.

But homicides don’t count all killings. So-called justifiable homicides don’t count toward the FBI definition. That category includes killings deemed to be in self-defense as well as the vast majority of the more than 1,000 annual killings by police officers, which usually are ruled justifiable — often unjustifiably.

Many major police departments reportedly have massaged crime numbers. So even the numbers officially reported to the FBI are suspect. Departments run by both Bratton and Kelly reportedly have fudged numbers, as have those in Philadelphia, New Orleans, Chicago and Milwaukee.

Even when departments are honest about what’s reported to them, we have no idea what isn’t. Most people who report being victims of crime in surveys don’t report them to the police.

Many other crimes also go unreported, and the extent to which they aren’t being counted can depend on local conditions. For example, the average response time by the New Orleans Police Department to calls is 73 minutes, leading some to question how many victims of property crimes don’t bother to report the theft. And most gunshots detected by an audio system installed by the NYPD didn’t get reported to 911. A study of the same technology in Washington, D.C., also showed a big divergence between gunfire and 911 calls reporting it.

The problem of underreporting is particularly pervasive for rape. And the rate at which rape is reported to police may not be an indicator of underlying trends. Reports of sexual assault rose in New York last year, a change which Bratton attributed to what he called the “Cosby Effect.” By that he meant that victims were coming forward about rapes from decades earlier, inspired in part by the dozens of women who accused Bill Cosby of rape, sometimes alleging the assaults happened decades earlier. About 1 in 5 rapes reported to the NYPD last year happened in an earlier year. But it’s impossible to know if that’s really behind the rise in sexual-assault reports: The NYPD only started counting delayed reports of rape last year.

CORRECTION (Jan. 21, 7:45 a.m.): A previous version of this article incorrectly described the change in crime statistics in Seattle in 2015. The number of crimes was 10 percent lower in the city in 2015 compared to the previous year; it was not up by 10 percent. The error was based on incorrect figures posted by the Brennan Center for Justice, and was brought to our attention by David Kroman, a reporter for Crosscut, a website that covers the Seattle area.

Jeff Asher contributed reporting.

Read more:

Murder Rates Don’t Tell Us Everything About Gun Violence

Scare Headlines Exaggerated The U.S. Crime Wave

Which Cities Share The Most Crime Data?


  1. It’s unclear how much this matters to the bottom-line numbers — the NYPD didn’t respond to my request for annual counts of grazings by gunfire and injuries from glass broken by gunfire.

  2. Through Dec. 23.

  3. More precisely, murder and nonnegligent manslaughter, per capita.

  4. R-squared, in other words.

  5. Wellington shared his unpublished analysis, which showed Spearman coefficients of 0.6 and 0.58 for the relationship between rank for murder rate and the ranks for assault and robbery, but 0.28, 0.31 and 0.4 for murder paired, respectively, with larceny, auto crimes and burglary.

  6. The average date for our counts was Nov. 24.

  7. As opposed to a police-department source or website, or a regularly updated online homicide tracker from the news media.

Carl Bialik was FiveThirtyEight’s lead writer for news.