In the first presidential debate, Donald Trump interrupted Hillary Clinton three times.
Turns out, a lot of people were counting, but nobody was counting in exactly the same way. As we gear up for the second presidential debate tonight, I thought it was worth exploring that, especially because we at FiveThirtyEight were a major outlier: We had the lowest interruption count, by far, of any news organization I could find.
|MEDIA OUTLET||COUNTED INTERRUPTIONS|
|Now This News||34|
|New York Times||39|
And reporters aren’t alone in our inability to settle on a standard way to count interruptions. After the Sept. 26 debate, Trump’s interruptions of Clinton became news because of the way they reflected both many women’s personal experiences and social science research that has, in general, shown that women get interrupted more frequently than men. But when I spoke with scientists who have done those studies, they told me that they haven’t been able to settle on a definition of “interruption,” either.
“It’s unimaginable how many definitions of interruptions I’ve seen in the [scientific] literature,” said Kristin Anderson, professor of psychology at the University of Houston-Downtown. “These are definitions that have to be operationally defined and submitted to peer review, and you still get a lot.”
It should be no surprise then that journalists — social science JV team that we are — counted interruptions in a variety of ways. Both FiveThirtyEight (lowest count) and Vox (highest count) went into the first presidential debate intending to track the number of times Trump interrupted Clinton, and vice versa, because of what reporters had seen of his style in the primary debates. Time magazine’s Chris Wilson, on the other hand, decided to track interruptions the next morning — after a mental light bulb went off at 6:30 a.m. and he came up with an idea.
“The challenge is that when you have an event that most people would agree is at least in part political theater, it’s difficult to come up with a way to quantify what happened,” he told me. Wilson’s solution was to use transcripts of the debate, provided by Congressional Quarterly, which inserted ellipses every time one speaker was still talking when another began. He used these to make a graphical interface that allows readers to skip through the debate interruption by interruption. In general, though, most reporters kept track during the debate, using predetermined criteria.
And that criteria explains the variation. Leah Libresco, a former FiveThirtyEight news reporter who joined us for the debate, chose to separately track interruptions — when the person who cuts in “keeps talking and winds up getting to be the person who talks” — and interjections — “attempts at an interruption that aren’t sustained.” She counted three interruptions but 24 interjections by Trump (and five interjections, zero interruptions by Clinton). In contrast, Vox’s Sarah Frostenson counted interruptions in a way she described as “very literal.” “There were some instances where Trump was just like, ‘Nope. Nope. Nope.’ Did it [in] three quick successions,” she said. “I think some people might say, ‘Oh, that’s only one interruption.’ But we took it literally.” Which led to her high-end count of 51.
And all three of these methodologies differ from criteria used in scientific research. For instance, in a 2015 paper that looked for gendered variables in several features of speech, interruptions were defined in a way that combines both successful and unsuccessful attempts to take over the speaking role and excluded something scientists call “back-channeling” — the “yeah” or “mm-hmmms” that we all toss out while listening (I kept back-channeling in interviews as sources talked about back-channeling) and which are generally meant to encourage the speaker to continue. Trump used these during the debate. There’s a “yeah” that shows up as a Trump interruption in Time’s count. It likely would have been counted by Vox, as well, based on what Frostenson told me about their methodology.
More complicated is an interruption toward the end of the debate, when Trump said, “Oh, really?” as Clinton began talking about Alicia Machado, the former Miss Universe who Trump has called fat. That could count as back-channeling. Or it might not. It depends on Trump’s intention. And this is really where counting interruptions — for journalism or science — becomes tricky.
Much of the diversity that exists in how social scientists define an interruption has to do with intent, Anderson told me. Adrienne Hancock, author of the 2015 paper and an associate professor at George Washington University, agreed. She told me in an email that intent matters more than the total count of interruptions. “However, it is challenging to measure someone’s ‘intent’,” she wrote. Conversation analysis — a specialization that derives from linguistics rather than sociology — tries to get around this by focusing on the effect the interruption had on the conversation and abandoning any attempt to guess at the speaker’s intent, said Celia Kitzinger, professor of conversation analysis, gender and sexuality at the University of York in England.
What scientists really want to know about, usually, is negative interruptions — when you cut in with the intention of taking over the conversation, not to encourage the speaker or agree with them. Back-channeling is a social science term and is often not counted in their studies of interruption because it is thought of as being positive and supporting … a sign of good listening. “But men are more likely to use back-channeling to take the floor,” said Deborah Cai, professor of media and communication at Temple University.
Context also matters. We might interpret the intention behind an interruption differently in a debate setting than in an office meeting, or at a bar with friends. (Cai and Anderson both said interruptions in a debate setting probably shouldn’t be looked to as reflective of real-world conversations.) We might also interpret the intention differently depending on who is doing it, and to whom. Case in point, the debate itself.
|MEDIA OUTLET||COUNTED INTERRUPTIONS|
|New York Times||9|
Clinton wasn’t the only one being interrupted. As all the outlets showed, she interrupted Trump, too, just much less frequently. (Libresco counted five interjections.) But Clinton wasn’t the person who was interrupted most frequently overall — at least not according to Time’s easily checkable numbers.1 That honor went to debate moderator Lester Holt, who was interrupted 27 times by Trump and seven times by Clinton. Meanwhile, if you watched a certain vice presidential debate, you probably also noticed two men interrupting each other … a lot.
That’s important to note because science isn’t clear on whether gender affects who gets interrupted. Kitzinger said conversation analysis doesn’t show an effect at all. Social science research does — but not as much as you might guess. In that 2015 study, both male and female subjects interrupted the scientist-trained conversation leaders they’d been paired with, and the rate of interruptions did vary by the gender of those partners. But differences between how frequently men interrupted women, women interrupted women, and men interrupted men were very small. The biggest outlier was in how frequently women interrupted men, which happened at a lower rate. Overall, the highest rates of interruption happened between two women.
|NUMBER OF INTERRUPTIONS|
|LEADER||INTERRUPTER||AVERAGE||MAXIMUM||PER WORD SPOKEN (BY INTERRUPTER)|
That’s not to say that women’s experiences are invalid or that women were wrong to identify with Clinton during the debate. Obviously, women are interrupted enough by men that this touched a nerve for a lot of people. It’s just that the science of how speech differs by gender is complex, doesn’t fit into a neat narrative and might not be the best way to capture what happens to women in the workplace. “It may be that powerful people interrupt less powerful people. Or that less powerful people feel interrupted or feel anxious about speaking. And in gendered workplaces, it’s quite likely that men are the more powerful people and women are the less powerful,” Kitzinger said.
And, ultimately, interpretation mattered to journalists, too. Despite the wide spread between their tallies of how often Clinton was interrupted by Trump, Libresco and Vox’s Frostenson interpreted their data in complementary ways. For Frostenson, the count was all about capturing the frequency and diversity of the interruptions — and she thought it was important to show how that validated the experiences of average women.
For Libresco, the count was all about capturing how frequently the interrupter was able to really take over. The low number of Trump interruptions, for her, demonstrated what Clinton wasn’t letting Trump do. “Where I am politically, there are things I like and don’t like about Clinton,” Libresco told me. “But having had the experience of being talked over and not being supported by a boss, it was such an f-yeah moment to see her not be cowed by a man who was trying to keep her from talking.”