In January, Philip McHarris was driving from New Haven to Boston to visit a friend when he saw a familiar sight in his rearview mirror — flashing police lights. It was late at night, and McHarris pulled into a gas station and waited, as he had done many times before, for the state trooper to approach his window. The problem, the trooper said, was the way McHarris had pulled off at an exit. Then he said that the highway where McHarris had been driving was a drug trafficking route, and asked if he could search the car for drugs.
McHarris explained that he was a Ph.D. student in sociology and African American Studies at Yale who had just left campus for the weekend. But while the officer walked away to his car, McHarris quickly took a video of himself and sent it to his mother and sisters. “I said, ‘This cop thinks I’m trafficking drugs,’” McHarris said. “‘I love you and I’m trying my best to navigate this.’”
Eventually, the officer let McHarris go with a warning. It didn’t spiral into one of the deadly encounters that make the front pages of newspapers, where a Black man is killed at the hands of a police officer. The officer had even been relatively courteous, assuring McHarris that the stop wasn’t the result of racial profiling. But that just reinforced for McHarris how poorly the officer understood the racial dynamics of the interaction — much less the fear he felt throughout the encounter, and couldn’t shake even after he drove away. “What good is it if a cop is being nice to me while asking to search the car?” he said. “What I care about is that I got pulled over in the first place, and I’m sitting here thinking maybe this random gas station is the last thing I’ll see.”
Years ago, McHarris came to the conclusion that because of interactions like this and his own research on policing, department-level policing reforms aren’t enough. And as a scholar who works alongside the Movement for Black Lives, McHarris is part of a small network of activists who have also spent years working to defund police departments, redistribute the money to other parts of the social safety net, like housing, education and transportation, and create new systems for ensuring public safety. But now their work is suddenly everywhere. After the police killing of George Floyd, the Minneapolis City Council voted to dismantle its police department amid growing calls from protesters to “defund the police.” And it’s not just Minneapolis — officials in Los Angeles, Denver and Portland, Ore., are mulling similar ideas.
[Related: The States Taking On Police Reform After The Death Of George Floyd]
The exact meaning of the slogan varies a lot depending on who you ask. Even the Minneapolis city council members who voted to disband say their move comes with qualifications. But broadly, it involves a seismic shift in the way we think about public safety — and who is being kept safe. Some critics of the defunding movement have argued that getting rid of the police would be counterproductive — in their absence, who would keep the streets safe? But McHarris said it’s time to stop tinkering around the edges of a system that many people in heavily policed communities say is causing more harm than good. “These police reforms are implemented over and over again and Black people are still being brutalized and murdered,” he said. “Nobody is saying we can’t have mechanisms to promote safety. It just won’t look like the police.”
Defunding the police is a big departure from the reforms we’ve seen before. But although there are disagreements between activists and researchers about how sweeping change should be, pretty much everyone we spoke with agrees that the system is broken, efforts to measure it are highly flawed, and now is the moment to think big about how to fix it. In many ways, the movement to defund the police is exposing gaping holes in how we measure what good police work really is, and how we gauge a reform’s success. Because after decades of research on policing and police reform, we still don’t know that much about what police are doing, how their presence actually affects the people who experience police violence, and what people in those communities want from reform.
On its surface, large majorities of Americans support “police reform.” But “reform” is vague and gets complicated fast. For one thing, the police aren’t a single entity. There are more than 15,000 law enforcement agencies scattered throughout the U.S., which means that any change has to be piecemeal. And it’s also hard to figure out what departments are actually doing, or how to compare them. Within a single metro area, multiple departments could be operating under different rules or different standards of rule enforcement, and even using different definitions of particular buzzword-heavy reforms like “community policing.”
[Related: How Americans Feel About ‘Defunding The Police’]
That lack of uniformity makes it difficult to compare police departments that have implemented similar policies. “To understand if a police reform is actually working the way you want, you need to be able to see what officers do in the field and figure out whether the reform you’re looking at changed that,” said Emily Owens, a criminology professor at the University of California, Irvine. “We don’t really have the data or the studies right now for me to say with confidence, ‘We know that these reforms work and these don’t.’”
What’s more, the data that exists is full of holes — and bias. Even when researchers try to document whether the police are doing a good job or how departments might improve, they’re often conducting those studies using metrics that help tell only part of the story. Policing data is imperfect. Due to a lack of systematic or reliable data on police misconduct, the fact that the data we do have is mostly from police departments themselves, and an emphasis on crime and police presence, it’s liable to miss important variables such as nature of police interactions with the public, or the fact that plenty of illegal or violent behavior happens in places and populations where police aren’t looking for it.
Case in point: The practice of hot spot policing. This is one of the best researched policing techniques and — after some 40-odd randomized controlled trials, by one expert’s count — also the one with the most evidence supporting its effectiveness. The basic idea is that crime is clustered throughout a city or neighborhood, so police should target those areas that see higher levels of crime.
But there are still a lot of unavoidable caveats. For instance, when scientists identify hot spots and measure whether policing in those locations has been effective, what they’re really looking at is crime statistics, said Cody Telep, a criminology professor at Arizona State University. That isn’t necessarily the same thing as measuring safety (real or perceived) in a community, he told us. After all, about half of all violent crimes are never reported to the police at all. So the appearance of a hot spot in the data doesn’t necessarily mean that’s where the most dangerous criminal activity is actually happening — something Telep’s team saw firsthand when it compared the locations of drug-related calls to police in Seattle with drug-related calls to the emergency medical services in the same city. It turns out there was a lot of drug activity the police were missing entirely.
Crime, then, and particularly crimes reported to police, are not a great metric by which to judge where the most crime in a city is happening and how dangerous that area feels to the people who live there. Nor is a reduction in crime the end-all, be-all metric to tell you whether police are doing good work. “Certainly reducing crime is a good metric, but I would add to that, at what cost?” said Rod Brunson, a professor at Northeastern University’s School of Criminology and Criminal Justice.
These statistics also don’t tell researchers anything about what police should be doing in a hot spot — or what police do in communities every day. Researchers have established that it’s effective to spend more time in certain parts of town … but, Telep said, have offered little guidance on what police should do once they’re there.
And even if police were given guidelines on what they should be doing, nobody currently keeps track of metrics that would show what those officers were actually doing. That’s about to change, at least partially — under an executive order issued by President Trump last week, the Department of Justice will begin maintaining an anonymized database of police misconduct. But there’s still a catch: The only time we find out about what police do in their day-to-day work is when someone complains or files a report about it, said Wesley Skogan, professor emeritus of political science at Northwestern University. That means there’s no incentive toward good behavior — even though some research suggests that positive interactions with police can improve public attitudes toward them. “The fact about policing is it’s two people in a car, in the night. What we know about what they do is when they choose to fill out a form,” Skogan said.
[Related: Republicans And Democrats Agree On The Protests But Not Why People Are Protesting]
But even in the situations where it is possible to get good data on real-world police behavior, whether a reform has been successful depends a lot on your perspective. In general, research tends to focus on metrics that are easy to quantify: Did a reform lead to fewer police killings? Fewer civilian complaints? More fired officers? But those measures don’t really account for the human and social cost of police violence, and they don’t tell us much about whether people in overpoliced communities are actually feeling safer.
Part of the problem is that we don’t have a good way to measure or track the effects of dealing with the police on an everyday basis. But qualitative research can give us a window into how a heavy police presence can stoke feelings of deep mistrust. In a study of young people in Baltimore conducted shortly after the killing of Freddie Gray, Yale sociologist and law professor Monica Bell concluded that although the people she was studying were very concerned about violence in their communities, they didn’t see police as protectors. “It’s not just that people are being brutally beaten or shot and killed by police, it’s the routine, daily messaging of — they are going to be watching you in your neighborhood, they’re going to mace you at your school,” she said. “People will even report that a police officer behaved in the ‘correct’ manner but they still walk away with a deep sense that they’ve experienced a broader racism, a broader sense of exclusion that quantitative measures can’t easily capture.”
Analysis of conversations with people in heavily policed communities by a group of political scientists found a similar result: To the people in the study, the police seemed like they were everywhere — except when their help was actually needed. The deep-rooted perception that the police are there to monitor you, not protect you, is hard both to measure and undo. “There’s a very strong perception that the police are there to protect and serve the white community on the other side of the city or the suburbs,” said Gwen Prowse, a Ph.D. candidate in political science and African American studies at Yale and one of the study authors. “The idea is — if you only see me as a criminal and you don’t see me as a full member of this society, how can you protect me?”
And without input from the people who actually experience police violence, attempts to evaluate police reforms can end up reflecting what the researchers — and not the people who are affected — think is important. “There’s this weird business where people think data can solve everything, but data without thoughtful engagement with the community is actually part of the problem,” said Bocar Ba, a postdoctoral fellow at the University of Pennsylvania Law School who studies economics and crime.
Use of force is one such example — just as police departments have different definitions of what constitutes “excessive force,” so can researchers. For instance, the idea that there is an “optimal” or “reasonable” use of force may prompt researchers or policymakers to ignore or discount lower-level incidents that were still quite traumatic for the person involved. “Who are the people deciding what an ‘optimal’ use of force is?” said Ba. “Are they the people who are experiencing brutality themselves? In almost all cases, no.”
Meanwhile, researchers are just starting to scratch the surface of how an uneasy and often violent relationship with police shapes other aspects of people’s lives. A recent study of students in an urban Southwest school district found that proximity to police shootings harmed high school students’ performance. That’s in line with other research indicating that having police in schools may actually decrease high school graduation rates. But these types of social costs aren’t easily priced in when researchers evaluate whether a reform succeeded or failed.
It’s becoming increasingly clear, too, that the reforms we’ve already tried are running headlong into other difficult to measure, difficult to fix forces, like police culture. Encounters between police and civilians are often violent because police officers are taught to think of themselves as always being in danger, according to research by sociologist Michael Sierra-Arévalo of the University of Texas. At the same time, Skogan said, police culture tends to discount things people at community meetings say they are actually interested in — like controlling traffic or reducing public drinking — as boring.
[Related: How The Police See Issues Of Race And Policing]
These are the kinds of problems that make activists like Arissa Hall, the director of National Bail Out, argue that simply reducing contact between police and civilians, and replacing the police with other community resources, is a much better way to address police violence. There might not be many precedents for disbanding police departments, but the positive effect of reducing police presence and investing in housing and education can already be seen in other places. “Abolishing police departments might seem impractical, but there aren’t police officers on every street corner in affluent white neighborhoods,” she said. “We have actual models and examples of what it means to decenter the police and invest in people’s quality of life.”
There’s a lot of research to support the idea that putting more money into resources that improve people’s lives — like health care, housing and education — can reduce crime. The more nebulous question is how removing funding for police departments will affect public safety. But Jennifer Doleac, an economics professor at Texas A&M University who studies crime, suggested policymakers and researchers should prioritize listening to the experiences of people who are interacting with police and make sure those are taken into account when deciding which policies are implemented and how to evaluate their success.
“Clearly, the system that we have now isn’t working for a lot of people,” Doleac said. “So can I see risks to significantly rolling back funding for police departments? Sure. But there’s also a big, big risk in doing too little right now.”