This week on Sparks, FiveThirtyEight’s monthly science podcast that runs in the What’s The Point feed, our science roundtable discussed Michael Lewis’s new book “The Undoing Project,” about two Israeli psychologists who created the field of behavioral economics. After that discussion, FiveThirtyEight editor-in-chief Nate Silver spoke with Lewis about the book, what it has to say about the 2016 U.S. election and Lewis’s writing process. You can listen to that interview above, watch a segment of the interview below, or read an abridged, lightly edited transcript below the video.
Nate Silver and Michael Lewis on the 2016 election
Nate: I think we’ve managed to talk to each other maybe once every couple of years, and I remember there were kind of seeds of this idea before, but what brought you to choose this topic and to write about Daniel Kahneman and Amos Tversky?
Michael: I wonder when I first mentioned it to you, because I first even heard of their existence after “Moneyball” was published.
Nate: Yeah.
Michael: So, in the year after “Moneyball” was published [2003], the behavioral economist Richard Thaler and the legal scholar Cass Sunstein published an article saying that in “Moneyball,” I kind of missed the story. They suggested that I should go read the collected works of Tversky and Kahneman to understand why baseball scouts misjudge baseball players. And I’d never heard of them, so I went and kind of started looking into it, and then found out that Kahneman was up the hill from me in Berkeley and went and visited him, and by about 2008, 2009, I kind of realized there was a story in the relationship. And it just took a long time to figure out exactly how to tell it.
Nate: Yeah.
Michael: And to gather the string for it. I mean, I had to spend a lot of time in Israel. I mean, the backdrop of this thing is Israel during the birth of the nation.
Nate: So, I’ve never been, but I like the kind of frontier —
Michael: You’ve never been?
Nate: Well, now I want to go after reading the book. So, I liked the frontier spirit where it’s just kind of a new country and it’s relatively hospitable to new ideas….
Michael: It’s like more Silicon Valley than Silicon Valley.
Nate: Yeah, yeah.
Michael: It really is. And also, it was a place where even the most ethereal, academic psychologists felt the need to pitch in and do practical things. So you have Danny Kahneman — who I think is probably by nature a French intellectual — but ends up redesigning the Israeli army’s personnel system. So, he builds the algorithm to determine who is going to be an officer and is training Israeli fighter pilots and tank commanders, so they’re mixing it up in the world in all kinds of ways that actually informs their work. Anyway, your question was, like, how did I decide to write about these guys, and there were a couple things going on. One, the ideas are fascinating. Most of the ideas, I bet, were already familiar to you. You probably were familiar with their papers.
Nate: Well, this is why it’s kind of strange to hear you say that you hadn’t heard — because it’s so implicit in so many things you’ve written about, right? Kind of errors in judgment and people from the outside detecting those errors in judgment, sometimes finding ways to profit off them in different ways.
Michael: Right. It’s a wonder that I hadn’t heard of them before. I suppose that’s right. There are huge gaps in my education and this was one of them. So, I came to it with a freshness, and that probably helped. If I thought, this is something I’ve kind of always known about, I probably would have been less enthusiastic about it. But what really charged me was the attempt, the possibility of writing this weird love story and taking these two guys’ ideas and the generation of the ideas and roughing it up against their consequences in the real world. So, it was partly a literary challenge. And I thought it was doable. My touchstone with books is always, are the characters good enough? And they were such good characters that I thought, this will work. I can make this work. And I did think at some point — and this is how I sold Danny Kahneman on the idea of letting me do this — that they’re so important, somebody one day is going to feel the need to do this book, and it might as well be me. Actually, my line to him was, “Someone’s going to do it, and it’s going to probably be done badly, and if anybody’s going to get a chance to do it badly, you should just let me do it badly. Because I know you now.” I mean, I got to know him over years.
Nate: Yep.
Michael: And Amos Tversky’s son was my student at Cal, and the Tversky family kind of opened up the archives and their lives to me in a really generous way, and it became pretty clear at some point — no one else was going to have the access. And what was going on while I was working on the book? The story was dying off before my eyes. I mean, every time I go to Israel, someone I interviewed the time before who was really, really important to talk to had died. So, it just kind of had to be done now or not at all.
Nate: And it has a very intimate feel to it, the book. I mean, it really kind of gets inside this relationship with stories that I don’t think had been told before, or at least not told outside of family and friend circles. How much time did you spend talking to Danny in particular? It was over several years?
Michael: Yes, the relationship was so opaque to people outside of the relationship. People — even people who you might think know more about these guys more than anybody — didn’t even know they broke up. They disguised the fact that they had a huge falling out from everybody. And I had their breakup letters. They were in Amos Tversky’s filing cabinet, so I could reconstruct the whole thing. And then Danny ended up helping, and I got help from Amos Tversky’s wife.1
But in the beginning, the first thing he said to me — I walked through his door in 2007, and I said it’s an honor. He looked at me strangely: It’s an honor? He said, “Oh, you mean the Nobel Prize. Never mind that. It’s not that big a deal.” And he tried to put me at my ease right from the beginning, and then involved me in the writing of his book “Thinking Fast and Slow” [2011]. So, he would send me pieces of it, I would read it and tell him that it wasn’t as a bad as he thought it was — in fact, it was really, really interesting. And then, I did things with him. I went to his last lectures at Princeton when he was a professor, and I helped teach his precept after the class. I went with him to Israel and we went and visited the Israeli army base. But we just did lots of stuff together.
I got bits and pieces over hundreds of encounters with him, either email encounters or long hikes or lunches or dinners or whatever. And it was all kind of by indirection, and it wasn’t until the very end, in the last year of so where I was saying, this is a book. You need to explain to me what happened in the chicken coop when you’re hiding from the Nazis. You’ve got to remember.
“I don’t remember,” he would say.
“Well, did this happen?”
“Oh, no, that didn’t happen.”
So, piecing his story together was, it was a bit like taking a vase that had been shattered and the pieces were all over the room and you’ve got to gather up all the pieces and try to glue them back together. And some of the pieces were missing, but he did his best to help me find them and then he’d show me where they fit.
Nate: You could read this book as having — at least in the interpersonal story — a slightly tragic ending. I don’t want to spoil it for people exactly. But it is different than, I guess the stereotype of the “Moneyball” type of book. Because in those books, you have an outsider who applies new ideas, and there’s some measure of vindication at the end. The A’s win the pennant, for instance. Or in “The Big Short,” some people are very right and make a lot of money and aren’t quite sure how to feel about that. But here, the payoff kind of comes sort of implicitly after the story ends.
Michael: In the payoff, there are a couple of moments where we break away from the main characters to show the consequence of their ideas in the world. And that’s a kind of victory. That is a victory. That’s what’s astonishing, the way the ideas seeped into everyday life. But the tragedy of the ending of the relationship … I mean, it’s poignant. These two guys, who really had more to do together, break up like lovers break up. And I’ve had a friend of mine, who the book is dedicated to, Dacher Keltner, who is a psychologist at Berkeley — really interesting guy, he’s the guy who introduced me to Danny — he said, you know, he was shocked when he read the thing and heard the story. He said, “Nobody does this in academia. There are lots of partnerships, there are lots of collaborations. They don’t fall in love with each other. They don’t break up. They don’t have this kind of dramatic arc. This doesn’t happen. You know, it just doesn’t happen.”
So, it’s a peculiar story in that way in that the intellectual interaction generated this sort of very high-pitched emotion. That is what attracted me to it in the first place. I mean, I can remember moments when I thought, “Jesus, this is a story.” And one of them is when Danny’s, like, dewey-eyed talking to me one day. We were just talking about whatever we were talking about, and he starts talking about how he felt about Amos. He says, you know, you’re in love with women and so on and so forth, but with Amos, I was rapt. And he said, and he cared more about me than I cared about him. And it was just the most important relationship in either one of their lives, and their wives even knew it. And that — I just thought, you can work with that.
Nate: I want to shift a little bit. This book came out almost exactly a month after the 2016 election. I know David Leonhardt of The New York Times said the election was a victory for gut instinct over empericism, for cynicism over reason. First of all, do you believe that? Do you have any sense for what Kahneman and Tversky would have thought of Donald Trump?
Michael: So, I’m not sure my thoughts are that interesting on this subject, but I am interested in filtering this whole election through their lens, because their work and I think they themselves would have a lot to say about what just happened. In no particular order, I think that they would say, “Our work is all about human fallibility. We search for error because the error gives us a guide, a window into the mechanism. The errors the mind makes tell us about the way the mind is. And we have discovered and we’ve shown over and over that fallibility’s just part of human nature and it’s not something to be ashamed of — it’s something, in a way, to be embraced and understood, and maybe corrected for. To have someone running for president who essentially insists he’s infallible is such a sign of stupidity. I mean, that kind of attitude to your mind and your gut instinct is idiocy. It’s stupidity. It’s not intelligence. It’s not a strategy. I mean, it is a strategy by default, but it’s not good.” So, they would look at Trump and be appalled, but then a lot of people do that. They’d be appalled just intellectually, never mind [his] groping and whatever he’s done with his taxes and all the rest.
Nate: Sure.
Michael: So, then I think they would say, “People are drawn, people want to make the world a more certain place than it is. They’re very uncomfortable thinking probabilistically. And they’re very uncomfortable turning to someone for advice or leadership and having that person be at all diffident, at all unsure. They want that person to seem totally certain. So, they want, in a weird way, idiocy from the people who give them advice. Stupidity. They want their financial adviser to say, buy IBM, it’s going up.” Now, what Amos and Danny would say is that if anybody says that, you should run the other way. Right?
Nate: Yeah.
Michael: However, people don’t do that. And so I think in a weird way, the people who were following Trump were the people who would take bad financial advice or bad medical advice or want a general manager who was totally sure that the guy he drafted was going to be a superstar in the NBA. All that. Trump was in a weird way preying on this need for certainty and confidence.
Nate: And uncertainty is a tough sell.
Michael: Right? I mean, it is a tough sell. That’s why this book is available to be written. That’s why you have a place in the market. There’s a kind of arbitrage going on. You’re arbitraging human nature against reality, and so it’s — you’re right. It is a tough sell. The quote in the front of the book sets the tone for the book. It’s from Voltaire’s letter, when he says, doubt is not a pleasant condition, but certainty is an absurd one. And I do think you’re right, uncertainty is a tough sell. You found it in your business to be so, but it is not an impossible sell, and the world’s moving in that direction. Haltingly, in fits and starts. Trump is a fit. Trump is a backwards step, but in the same way that baseball management has become less and less about the raw, intuitive judgment of so-called experts and more and more about data based decision making
Nate: But do you think there’s something about politics in particular? Because baseball … there has been kind of a revolution, and it turns out that it’s not a unilateral win by the geeks, but you know, in every front office — or 29 out of 30. Whereas in politics, you take an example like Brexit, where the polls showed a too-close-to-call race and the pundits said no, no, no, no, it’s going to be Remain, Remain, Remain. And then Leave wins, and then the pundits blame the polls.
Michael: Why isn’t politics more like baseball? Well, political management, campaign management has become a bit more like baseball. More data driven. And just because Hillary Clinton lost and was supposedly a more data-driven operation doesn’t mean that the process of managing the campaign was stupider. Maybe you just had a product that wasn’t as desirable. I mean, no brilliant geek in a baseball front office is going to be able to beat a really good opponent with a bunch of triple-A players. Numbers will help you understand your predicament, but you can’t completely change things. And having said that, I’ve read quotes from the Trump children and the Trump son-in-law saying, we just did Moneyball for politics. That we Moneyball-ed the Electoral College.
I don’t know what they mean by that, but they clearly were not hostile to the idea of being analytical about how they approached their campaign. So, I’m not sure there’s a message in that. It is true that, you know, people have trouble accepting that polls are only rough guides to what the population is going to do. And the analysis of the polls — as good as it is — if you only have a rough picture of the electorate, the analysis is not going to give you the perfect picture. And there’s going to be some unpredictability in it all.
Nate: And people are also kind of anchored to the most recent election, where in 2012, you had polls, and in 2008, for that matter, you had polls that were both very stable and very accurate. You don’t have to go back that far to years like 2000 where there’s a fairly big polling error, or 1980, or 2014, if you look at a midterm.
Michael: And also, if you took what just happened and switch a few thousand votes in a few states, the narrative would be completely different.
Nate: Of course.
Michael: What’s more interesting to me about the response to the election is what people want to say about it. How many people want to instantly analyze. What’s the substitute? I mean, I do feel like, it’s like the person who walks into the casino and is given a choice between giving their money to someone who says they feel lucky at the blackjack table and giving their money to someone who is a provenly good card counter. Now, sometimes the person who feels lucky will win. Actually, quite a bit. They’ll win, but they won’t win as much as the card counter. But it’s like taking the moment after the guy who feels lucky wins a hand and saying, look, all that card counting’s bullshit. It’s just dumb. But it rises from some deep place. People don’t like — especially people whose job it is to be the expert — being challenged.
Nate: Yeah, because this book is — among other things — and Kahneman and Tversky’s work is a critique of expertise, right?
Michael: It is indeed. It is indeed. And it has a lot to say about financial expertise and political expertise and medical expertise and sports management expertise, and it’s skeptical of expertise. It’s saying that — not in a hostile way, it’s not singling out the experts. It’s saying that experts, like everyone else, are wired for certain kinds of mistakes, and they will make them.
Nate: I want to take a few final minutes here and talk about your process as a writer. And this is a little bit selfish because I wrote a book myself a few years ago and found that there weren’t a lot of people to talk to about what it’s like to write a book. But one thing I was interested by is you don’t take — this is a little inside baseball, but seems appropriate given the topic — you don’t take advances for your book.
Michael: No.
Nate: Because you feel — is that, like, deliberately because you think it would skew your incentives?
Michael: Yes. I think people don’t pay enough attention to the incentives they bake into their lives, and they think they’re more immune to them than they are. And I think that, I don’t know, if they take a big advance, oh, it’s not going to matter. It’s just smart strategy. I think that people should own pieces of their companies. I don’t think anybody should be just an employee, unless you’re really not there, unless you don’t intend to be there for very long. But if you’re really going to do a good job, it helps to be an owner. And I want to own my book. I mean, I want to own the process. I want to know that I have risk. I want to know that if it stinks that I’m not going to make any money. And so I like having skin in the game. Actually, I don’t like it. It makes me uncomfortable. But I think that discomfort is of great value, and that it forces me — even if I’m not really thinking about it. As I’m writing a book, I’m not thinking I didn’t take an advance for it. I forget about it. But somewhere deep down, I’m incentivized to work a little harder. So that’s why I don’t take an advance. I want to be exposed to the fortunes of the book.
Nate: One of the parts that felt uncomfortable to me in reading the book was when Kahneman and Tversky were collaborating on writing papers and literally had to agree on every sentence. I mean, when you’re writing, is it a solitary endeavor? Are you in a coffee shop somewhere?
Michael: No, I shut the world out. Totally. And the way I shut the world out, I have an office. It’s its own building, and I have to eliminate even the need, even the possibility of distraction. So there are no phones or anything like that. And I create a playlist for each book. I pick like 15 pop songs, and I play them on a loop. And what happens is I cease to hear them or anything else, and it ends up being this like Pavlovian response to the songs. When I hear the songs, it’s writing time. And it is, it really is amazing how after a little while, if I’m wandering down the street and I hear one of the songs, I’m instantly looking for my laptop and thinking I should write. But that’s what — I shut the world out, and it is solitary. Not in an artistic way.
Nate: Do you tend to write until you get exhausted, or is it more regimented and say, I’m going to write from 9 to 5 today?
Michael: It’s not on a clock, usually. If I’m in a pinch and I’m afraid that I’ve actually kind of fallen behind a deadline — I need deadlines, so I always give myself deadlines. And oddly, even though the deadlines are always kind of artificial and self-imposed, they assume the draconian aspect of a real deadline. I believe my own lie. And the way I measure myself is just how many words I put on the page. So, I’ll say to myself, I’m not getting up until I’ve written a thousand words. Just not getting out of the seat. So, once I’ve told myself that, then what happens is once you create those pressures, you actually forget about everything and you just write. And, you know, more comes out. The goals that I set become minimums, and I’m off in my own space and free to work. I’m basically lazy. I mean, really. Basically, I would do nothing if I didn’t create these artificial structures to prod me.