The first time Bob Duffy entered the world of epidemiology, he was an amateur scientist. It was 2003. He had retired from the New York City Fire Department and taken a sabbatical from his normal life in suburban Long Island to help his daughter Meghan earn her Ph.D. in Michigan. She was studying the ecology and evolution of infectious diseases, using tiny lake crustaceans as a model organism.
Together, Meghan and Bob would go out in a truck, towing a little, flat-bottomed rowboat. They were studying how epidemics begin and spread under a variety of conditions. They’d unhitch at one lake, and then another, working their way across the countryside as they collected and counted diseased crustaceans and the fish that preyed on them. “Over the course of a few months, you can go through a whole epidemic,” Meghan Duffy told me. Her father was her paid research assistant, and one of his jobs was to catch the fish. After 30 years of running into burning buildings, he couldn’t believe his luck, she said.
The last time Bob Duffy entered the world of epidemiology, he was a statistic.
Bob Duffy was a father, grandfather, retired firefighter, and longtime volunteer in his Long Island community. He died on March 29.
COURTESY OF MEGHAN DUFFY
He died, at home, on March 29, 2020. Officially, the cause of death was chronic lung disease. But there was more going on than just that. A sudden illness had left him too fatigued to leave the house, and he had had contact with multiple people who later tested positive for COVID-19. Yet Bob’s death certificate doesn’t list that disease as a cause or even a probable cause of his death. He never got tested — he didn’t want to enter a hospital and be separated from Fran, his wife of 48 years.
Instead, because he didn’t die at a hospital and because this was at the beginning of the pandemic, when guidelines were rapidly changing and testing was hard to come by, Bob Duffy became one of the people who fell through the statistical cracks. As of this writing,1 22,843 New Yorkers have officially died from COVID-19. Bob Duffy is not counted among them.
More than a month later, the question of who counts as a COVID-19 fatality has become political. In Florida, the Medical Examiners Commission accused state officials of suppressing their state death count. Pennsylvania’s death tally bounced up and down, enough to prompt the state senate to discuss giving coroners a bigger role in investigating COVID-19 deaths. And President Trump has questioned the official national death count of 90,340 as of May 19,2 reportedly wondering whether it was exaggerated.
The experts who are involved in counting novel coronavirus deaths at all levels — from local hospitals to the Centers for Disease Control and Prevention — disagree with the president. If anything, they say, these deaths are undercounted. And with a death like Bob Duffy’s, you can begin to see why.
Bob was a person, beloved by his family and his community. Ever since he died, Bob has also become a number — data entered into a spreadsheet, just like the tiny shellfish he and his daughter once pulled from cold Michigan lakes. His death might never end up being attributed to SARS-CoV-2, but his death matters to the way we understand it.
There was never a cough. Instead, the first sign of illness Fran Duffy remembers was when she and Bob tried to go for a walk and he couldn’t make it to the end of the block. “We got three houses down, and he said, ‘I can’t walk today. I’m too tired.’ I thought maybe he’s getting a bug. Maybe he’s just tired. So we came back. That was Wednesday,” she said.
He died four days later.
It was a very fast decline. But in other ways, Bob’s final illness was just part of a long string of sicknesses. Over the two decades since his retirement, he had had a stroke. He also had had cancer in his mouth, colon and liver. There was scarring — fibrosis — that had damaged his lungs and forced him onto supplemental oxygen. The radiation treatments that had cured his cancers years ago had also left him with nerve damage in his legs and a slowly eroding jawbone. Bob was not the picture of health. We are, after all, talking about a guy who worked for the NYFD during a time when firefighters did not routinely wear the ventilators and masks they had been issued. It was a macho thing, Fran said. You couldn’t be the one guy who put on the mask if nobody else did.
So when Bob got sick in late March this year, whatever it was was not the only thing he was sick with. He was also so sick of being sick that he wasn’t interested in going to the hospital. Even as his temperature soared to 103 degrees, Bob chose to do a video chat with his family doctor, Ihor Magun, rather than leave the house. Fran remembers the doctor suggesting they treat Bob as if he was positive for COVID-19, in terms of isolation from friends and family. He could have gotten a test — but the nearest testing center at Jones Beach was 30 minutes away, and then there were the long lines besides. Fran thought about driving him out there, but he was already sick enough that that option seemed worse for him than not knowing what it was that he had contracted.
All those small decisions, made in the moment because of what was best for Bob, ended up determining how his death was recorded.
The way deaths are counted, like so much else in the U.S., differs among (and even within) states. There’s a lot of variation in this process, even on a good day — a fact that stretches all the way back to the beginning of mortality records in this country. While the census began counting living people nationwide in 1790, recording deaths was left up to state and local governments. The first state to fully document its deaths was Massachusetts, in 1842. It wasn’t until 1933 that all states were turning in death counts to federal authorities.
Even today, now that the death certificate itself is fairly standardized, who first records your death and decides what you died of varies by where you live and where you die. And that variation is only likely to increase when people begin dying of a new disease that we still don’t understand. In Milwaukee County, Wisconsin, for example, medical examiners — medical doctors who investigate deaths and perform autopsies — must provide official certification for every COVID-19 or COVID-19-related death in the county, said Dr. Sally Aiken, president of the National Association of Medical Examiners. But that’s not true everywhere. In New York State, medical examiners get involved only in cases that seem strange or suspicious, like when an otherwise healthy young person dies with no prior warning, said Richard Sullivan, president of the New York State Funeral Directors Association. Otherwise, the decision is left up to health care workers.
Bob’s death certificate was filled out by his family doctor and did not mention COVID-19. The county medical examiner called Fran but asked only about Bob’s preexisting conditions. He had had enough of them that there was no reason to suspect foul play, and that was all the medical examiner needed to know.
If Bob had died in a nearby hospital, such as one of the ones in Nassau County owned by Northwell Health, he would have been tested for COVID-19, either before or after his death. Whether he’d been there for five minutes or a month, hospital staff would have been in charge of filling out the part of his electronic death record that pertains to cause of death, a representative from Northwell told me. This process can look deceptively simple — just write a cause of death on the line — but there’s more to it than you’d think.
A standard certificate of death provided by the National Center for Health Statistics leaves room for the chain of events that led to someone’s death.
The New York electronic death records form provides three lines for cause of death, which are supposed to be filled out in a way that tells a story. The idea is that nobody ever really dies of just one thing, Aiken told me. Even if you die in a traffic accident, the death record might read something like “Blunt force trauma … as a consequence of a car crash.” This is the information that helps people further up the data chain classify a death accurately. Leaving any part of the story out means a gap in the data later.
Not everyone fills out these records completely, though. And early on during the COVID-19 pandemic, there was a lot of confusion happening, said Shawna Webster, executive director of the National Association for Public Health Statistics and Information Systems, which represents vital registrars nationwide. “It might just say ‘coronavirus,’ which I’m sure you know is not as descriptive as it needs to be,” she said. There are, after all, multiple ways COVID-19 might kill a person. On the other end of the spectrum are people who fill out the forms completely wrong. “Please do not put ‘COVID-19 test negative,'” Webster said. “Do not do that. There were several.”
In the days after his first symptoms, Bob’s condition worsened. He’d become so tired he couldn’t leave the house — then so tired that walking anywhere by himself was impossible. He had a massively high fever. But even Saturday, the night before he died, he was still talking, Fran said, and so she asked him what he wanted for dinner. She expected something light. Bob said, “Corned beef hash.”
“I said, ‘Bob, corned beef hash?'” But he was sure. So Fran put it together for him, the man she loved. She had to move him to a wheelchair and bring him to the kitchen to eat. He could no longer walk without falling. “I bring him to the kitchen and I’m just turning to the sink to wash my hands and I hear plop,” she said. He had fallen asleep at the table. “His head went right down in the plate. And I just said, ‘Bob. What about the corned beef hash!’ So it just … he thought about it and he wanted it, but he just couldn’t get it, you know?”
Doctors say this kind of oxygen depletion and exhaustion — coupled with an ability to still communicate — is a common feature of COVID-19. Even after he collapsed at the table, Bob was lucid enough to talk to the priest who gave him his last rites later that night. He died the next day.
Over the next few weeks, it would become clear that Bob had been in contact with a number of potential sources of COVID-19 — or maybe he’d been a source that passed it to them. It’s impossible to know. His son-in-law was later diagnosed with the disease, and his wife — one of Bob’s three daughters — tested positive for COVID-19 antibodies. One day Fran would open the newspaper to find that the woman who had cut her and Bob’s hair for three decades — and who had come to their house just before Bob got sick — had died of COVID-19.
But Bob’s death certificate makes no mention of the novel coronavirus. Bob’s doctor did not return requests for an interview, so we don’t know why he made the choices he did when completing the certificate. But Bob’s immediate cause of death is listed as “cardiopulmonary arrest” — his heart stopped — as a consequence of “chronic obstructive lung disease,” as a consequence of “fibrosis.”
Bob is a prime example of why doctors and other experts think that COVID-19 deaths are probably being undercounted — not overcounted, as some COVID-19 skeptics have alleged. In fact, if Bob had died today, there’s a decent chance that he’d have been labeled a “probable” COVID death, based on current CDC guidelines, which, among other things, advise doctors to include “probable COVID-19” on death certificates when a patient has had symptoms of the disease and been in contact with people who tested positive. Originally, only people who themselves had tested positive for the virus were being counted. Like Bob, a lot of people were probably left out. But even as the guidelines were revised and the national death count — which includes probable as well as confirmed cases — shot upward, experts said that undercounting was still more likely than overcounting.
COVID-19’s death toll has been so overwhelming that officials have had to resort to makeshift morgues in trailers.
TAYFUN COSKUN / ANADOLU AGENCY VIA GETTY IMAGES
Some of this reasoning is based on logic. We know that we had a widespread shortage of tests when people were already dying of COVID-19, so it makes sense that these two problems would overlap at times.
Other reasoning is based on data. In a lot of states the number of pneumonia deaths in March was higher than what you’d expect for that time of year, or for the level of influenza active during that time — an important detail, given that pneumonia can often be a complication of that disease as well. These increases were particularly noticeable in New Jersey, Georgia, Illinois, Washington and New York, according to research led by Dan Weinberger, a professor of epidemiology at Yale School of Medicine. But pneumonia isn’t the only way COVID-19 kills. All deaths in the state of New York went up in March, and these excess deaths — deaths above the usual rate for that place and time of year — outstrip diagnosed COVID-19 cases statewide by nearly three times. Data collected by The New York Times suggests that the high number of “excess” deaths in New York continued through April.
Yet another reason why experts say we’re not overcounting COVID-19 deaths is that we’re now counting them in much the same way as we have always counted deaths from infectious disease. The methodology is longstanding and is used for all sorts of diseases — and there’s never been cause to think that the methodology made us overcount the deaths from those other diseases.
If you look at the CDC’s annual report of flu deaths, for example, you’ll see that it’s “estimated,” modeled on official flu deaths reported, deaths from flu-like causes reported, and what we know about flu epidemiology. The calculation is done this way precisely because public health officials know that a straight count of formally diagnosed flu deaths would be an undercount of actual flu deaths.
While flu tests aren’t in short supply and essentially anyone who wants to be tested for the flu can be, not everyone who catches it gets tested. Plenty of people get sick with the flu and never go to a doctor, said Alberto Marino, a research officer at the London School of Economics who has studied disease case and death counts for both LSE and the Organization for Economic Cooperation and Development. If they die — especially if they are also old or have some underlying condition — the role the flu played in their deaths can easily go unnoticed and unrecorded. We don’t record “probable” flu deaths (again, the tests aren’t rationed), but we do record deaths due to “flu-like illnesses” — and plenty of people who die from the flu don’t have that listed as the cause on their death certificates.
Likewise, when a doctor lists COVID-19 as a condition that led to someone’s death — even if it was just the last in a series of illnesses — they’re not doing anything different from what’s been done with the flu for years, Aiken told me.
Basically, if you think COVID-19 deaths are being inflated, then you shouldn’t trust annual flu death counts, either. Or a whole host of other death counts. The only reason to really think that COVID-19 death counts are less trustworthy at this point is that the flu is politically neutral while the new coronavirus is not.
If there’s any major difference between the way we count flu deaths and the way we count COVID-19 deaths, it’s that nobody is trying to publish flu deaths daily, in real time. And that’s where death counting for COVID-19 gets complicated.
When Bob Duffy died, his community responded immediately. Fran found her mailbox filled with cards; flowers and baked goods piled up on the porch. At one point, there were so many tulips, hydrangeas and pansies that the Amazon delivery guy started to make comments, so Fran decided to plant the flowers around the yard. “There’s not one card that doesn’t have a separate letter in it,” she said. And many were from people she didn’t even know.
Besides being a firefighter and Ph.D. assistant, Bob spent many years working with the local Catholic parish’s social ministry. Essentially, he was a volunteer social worker. He made sure people who were hungry found meals. He helped strangers pay their utility bills, and he coordinated a Long Island-wide food bank. “Most people volunteer one day a week. Bob officially volunteered five days a week,” Fran told me. “He ended up with the keys to the parish. He was up there seven days a week, and he couldn’t be stopped.”
So when he did stop, people cared. And they cared for his widow.
Bob Duffy’s family will never know for sure whether he died of COVID-19.
COURTESY OF MEGHAN DUFFY
Death happens suddenly, abruptly. At first, family, friends and, sometimes, if we’re lucky, strangers burst into action like Roman candles, sending out showers of casseroles and condolences like sparks. For a short period of time, there is a lot to do, decisions to be made, love to be accepted. But then there is quiet. And then there is the rest of your life. The absence that death leaves behind lasts far longer than the initial flurry of condolences.
The bureaucracy of death has a similar dynamic — first, everything happens fast, fast, fast, and then, after a while, things just grind on.
In New York, in the heady first day or two after a person dies, the doctor or hospital enters the cause of death on an electronic death record, the funeral home fills out demographic data on the same form, and the state registrar of vital statistics logs the data. But from there things slow down considerably.
Usually, that’s fine — death statistics aren’t so volatile that we need them to be updated as quickly as, say, election returns or live sports scores. But the pandemic has changed our relationship with these stats. Now they’re how we know whether we’re stopping the spread of COVID-19, and just how big that spread is. The problem is that the system isn’t designed to do that work.
Normally, if a death is uncomplicated and requires no investigation or autopsy or debate, death records are transferred to the National Center for Health Statistics, an arm of the CDC that organizes and analyzes the data of life and death in this country. It’s here that a death is categorized and tabulated. And this process is happening now, with COVID-19 deaths as well.
It takes time to investigate some of the deaths and get them to NCHS — the frequency of investigations varies widely, but state-level emergency operations teams work with medical personnel and state epidemiology surveillance to review COVID-19 deaths and possible COVID-19 deaths, Webster said. So the records can be in the state databases for a while before they’re solid enough that they go to NCHS. Then, someone at the NCHS is reading each of these death records to make sure that, say, a car crash victim who happened to have a COVID-19 diagnosis is logged in a database differently from a COVID-19-positive patient who died on a ventilator. The result of all this is that, even though public counts include confirmed COVID-19 deaths and probable ones, the deaths aren’t just being recorded willy-nilly. And it will be possible, in the future, to go back and look at the records and see which cases were confirmed by testing and which weren’t.
But these are slow stats. And they’re slowed down even further by the confusion caused by a novel virus pandemic. Currently, the count of COVID-19 deaths produced this way is at least two weeks behind, said Robert Anderson, chief of the mortality statistics branch of the NCHS. The counts in some states, including New York, might be lagging even more. This system is the gold standard, Webster said, but it’s designed to produce accurate statistics — not monitor a pandemic in real time.
And so the CDC also has fast stats on COVID-19 deaths. Besides going to the NCHS, the data from the New York State vital records office is also gathered directly from that agency’s database and into one maintained by USAFacts, a nonpartisan nonprofit organization charged with collecting daily death reports from the state and county registrars that first record them. The CDC’s COVID Data Tracker comes directly from the USAFacts count.
That means there are two distinct death counts being published by the CDC — one slow, one fast. (That’s in addition to counts being kept by Johns Hopkins University, The New York Times, and other entities.) As of May 19, the CDC’s slow count was 67,008, and its fast count was 90,340. You’ll find both counts in various sections of the CDC’s website, and when you look at those pages, it’s not always clear what these separate counts do and don’t represent. It’s easy to get confused and assume that the death count you’ve just seen in the newspaper has suddenly been cut in half. On May 2, conservative firebrand Dinesh D’Souza falsely claimed exactly that, linking his followers to the CDC’s slow count.
The smaller, slow count is more accurate, but it doesn’t reflect how many people have died as of today. It’s weeks behind. The fast count does a better job of portraying the real-time situation, but the exact number will shift as state and local counts fluctuate. Some of that change is due to confusion between state and local entities. New York City, for example, has its own vital records office — almost as though it’s an independent state — and the fast-count numbers it produces for itself don’t usually match the fast-count numbers produced for it by the State of New York, said Tanveer Ali, a data visualization analyst for USAFacts.
And while Bob Duffy will not be counted in either the slow or the fast counts happening now, he will likely end up included in the data — if only by algorithmic proxy. Eventually, experts said, the CDC will come back and do an estimated burden of death counts for COVID-19, just as it does for the flu every year.
All of this is why we won’t know the exact number of people who died of COVID-19 for years, Aiken said. Again, that’s nothing new. Final estimates for the number of people who died in the 2009 H1N1 pandemic weren’t published until 2011. Getting the slow count right, sorting through differences between disparate and nonstandardized state reporting systems, correcting errors and categorizing probable cases, finding ways to understand how many Bob Duffys we’re missing — it all takes time. This is, experts emphasized again and again, something nobody has ever done before. But the precedent that does exist suggests we shouldn’t expect to get a “right” answer soon. “If you look at opioid mortality, they’re two and a half years behind on compiling that,” Aiken said.
Death is hard — hard to count, hard to experience. The personal and the statistical both reside in a space where the question of “what happened” can be answered as an absolute — as certain as we can ever be about a thing — while simultaneously remaining painfully inexact and mysterious.
We will almost certainly never know exactly how many Americans died of COVID-19. But any count we get by leaving out deaths probably related to the virus — and, ultimately, leaving out Bob and a lot of people like him — will be less accurate than a count that includes them.
“We like to have answers. We like to have a yes, a no, a definite answer,” Fran said. Bob had been dead for about a month when Fran spoke to me from her kitchen. Just that day, someone she didn’t know had sympathetically left a loaf of banana bread in her mailbox. He was still so close. He was so far away. “But we certainly don’t always get what we like,” she said. “That’s really the truth, you know?”
Additional reporting by Kaleigh Rogers.
Subscribe to our coronavirus podcast, PODCAST-19