There are algorithms all around us. They suggest which route to take, what song to listen to next, and what articles we might like. The algorithms that author and data scientist Cathy O’Neil is most interested in meet three criteria: They are widespread, secret and unfair — the scoring methods that she says, for example, generate credit scores, keep someone in prison, or deny someone a job. On this week’s What’s The Point, O’Neil discusses her new book, “Weapons of Math Destruction,” which explores the world of algorithmic bias — and makes a call for something new: “algorithmic auditing.”
Below is a partial transcript of the conversation, including O’Neil’s discussion of how algorithmic screening can prevent someone from getting a job.
Jody Avirgan: You pointed out that usually you don’t know why you got [rejected], but even if you know that you didn’t get the job and get frustrated and want answers, it’s not like you can walk into the algorithm’s office and say, “Hey, what happened?” There’s an opaque chain, and no one is truly accountable. It must just be very frustrating.
Cathy O’Neil: And shaming. I think a lot of the examples that I have in the book have a weird characteristic that we don’t think about that much, which is that people are actually shamed when they fail and when they get bad scores. We often in this country associate doing well on “tests” with having some kind of moral uprightness. And I think this is an example where [people] feel, “Am I that broken?” That’s one of the reactions to failing a test. Like it’s an indictment of character.
Avirgan: Even more so than having a hiring manager say, “I don’t trust you,” or having a judge saying, “You’re at risk for parole violation”? When it’s an algorithm it feels more shaming?
O’Neil: You mentioned earlier that one of the problems with these algorithms is that they seem so neutral and objective and unbiased, and they are held up as so authoritative. That’s one of the reasons we give them so much power. But the side effect is that it’s not just “this guy didn’t like me” — it’s that “I failed this scientific measure of my aptitude.”
If you’re a fan of What’s The Point, subscribe on Apple Podcasts, and please leave a rating/review — that helps spread the word to other listeners. And be sure to check out our sports show Hot Takedown as well. Have something to say about this episode, or have an idea for a future show? Get in touch by email, on Twitter, or in the comments.