FiveThirtyEight relaunched less than two weeks ago. It’s been a heck of a learning experience. When you’re trying something new, it’s going to take some time to get everything right, and you’re going to get criticism from all quarters.
There are two types of criticism that we’re most concerned with. First is criticism of our execution: How well are we living up to the standards that we’ve set for ourselves? Second is criticism that comes from our broader community of readers, such as on Twitter, by email or in the comments section of our website (as opposed to criticism from other people in the news media).
One article on FiveThirtyEight has been disproportionately responsible for substantive criticism from readers. That was a piece we ran March 19 from Roger Pielke Jr., a freelance contributor to FiveThirtyEight and a professor of environmental studies at the University of Colorado Boulder. The story concerned the economic costs of climate-related disasters. The central thesis of the piece was that although these costs are increasing, the rise can be accounted for by the growing wealth of the global population, rather than by a rise in the number of disaster events due to climate change.
Reception to the article ran about 80 percent negative in the comments section and on social media. A reaction like that compels us to think carefully about the piece and our editorial process.
The responses have fallen into about four broad categories. I list these in order of most to least concern to us:
- Criticisms of Roger’s central thesis about disaster costs
- Concern about how FiveThirtyEight will be covering climate topics
- Criticisms of other claims Roger made in the article, such as those about the overall incidence of weather-related disasters
- Criticisms of things Roger has said or written in other venues, sometimes including ad-hominem attacks against Roger
Let me deal with category No. 4 first. Roger and his critics can kick up a lot of dust everywhere they go. Some of the criticisms of Roger have been unfair. For instance, Roger is not a climate “skeptic” or “denier.” He has written at FiveThirtyEight — and he has testified before Congress — that he believes in the thesis of anthropogenic global warming (AGW), that he considers it a serious problem, and that he thinks society should make efforts to mitigate it.
Another line of criticism is that Roger is unqualified to write about climate change because his training is as a political scientist rather than a climatologist. However, the scientific consensus on the climate — as embodied in the extensive list of Intergovernmental Panel on Climate Change (IPCC) authors — is formulated not only by climatologists, but also statisticians, meteorologists, engineers, economists, ecologists, physicists and those from many other disciplines in the hard sciences and the social sciences. Most have expertise within some relatively narrow part of the literature. Roger has published dozens of peer-reviewed articles on estimating the incidence of climate-related disasters and their associated costs. That was the subject of his FiveThirtyEight piece.
We’re much more sympathetic to the other three categories of criticism, however.
Criticisms of other claims Roger made in the article
As I mentioned, the central thesis of Roger’s article concerns the economic costs associated with natural disasters. But we also allowed a number of peripheral claims into the piece. For instance, Roger made a number of references to the overall incidence of natural disasters, as opposed to their economic cost.
We think many of these claims have support in the scientific literature, specifically including the 2013 IPCC report. But there is a range of debate among experts about others. Either way, these claims shouldn’t have been included in the story as offhand remarks. We should either have addressed them in more detail or scrubbed them from the article.
Roger’s article also contained an implicit policy recommendation in its closing paragraph. Whether or not the recommendation was justified by Roger’s thesis and evidence, we generally prefer to avoid these kind of recommendations, and instead allow readers to draw any policy conclusions for themselves. Furthermore, there was some loose language in the article. We pride ourselves on precise, matter-of-fact language. These things reflect a poor job of editing on our part.
Concern about how FiveThirtyEight will be covering climate topics
We understand the urge to make extrapolations about FiveThirtyEight’s overall coverage of the climate (or scientific topics more generally) from Roger’s article about disaster costs. We also recognize that even if Roger were right on the specific points he made — and obviously, some people dispute that — it’s possible to lose the forest for the trees.
However, the article is a sample size of one. Roger’s piece is not the complete story about climate change. Nor is it the complete story about how FiveThirtyEight plans to cover the climate. We didn’t hire Roger to write solely about climate (instead, we hired him to have a relatively broad portfolio, such as writing articles about sports statistics). Nor do we plan for Roger to be the lone person at FiveThirtyEight writing about climatology and climate data.
Criticisms of Roger’s central thesis about disaster costs
Roger knows the literature on disaster costs extremely well, and I know he disagrees with many of the criticisms about his piece. But some dissenters feel just as firmly. The back-and-forth is extremely detailed, citing paper upon paper, footnote upon footnote, and link upon link. The debate is hard for us to adjudicate without turning to experts for help.
John P. Abraham, a professor of engineering at the University of St. Thomas, is one scientist who has spoken publicly about his concerns over Roger’s article. He also emailed me privately, and I asked him to consider writing a response to Roger for publication at FiveThirtyEight. Abraham kindly submitted an article to us. However, after a discussion with him, our editorial team concluded that the piece wasn’t quite what we were aiming for. Its criticism spanned several of the categories I outlined above, when we were looking for something more focused on the scientific claims made in Roger’s FiveThirtyEight article. Instead, Abraham ran his article at The Huffington Post. We appreciate his understanding, and I’d encourage you to read his response. You can see even stronger criticism in another article written by Dana Nuccitelli, who is Abraham’s blogging partner at The Guardian.
Nevertheless, we see value in running a rebuttal to Roger’s article at FiveThirtyEight itself. So we are in the process of commissioning one from someone who 1) has not yet weighed in on Roger’s article and 2) has very strong credentials. The scientist who is our No. 1 choice is traveling, and so the turnaround will not be instantaneous.
We appreciate your patience in the meantime. Climate change is not going away as an issue, and we want to get this right. All journalism relies on trust — between reporters and sources, between editors and writers, between a publication and its readers. Any time that trust is undermined, it’s a huge concern for us. We thank you for your continued feedback. We’re listening and learning.
UPDATE (March 31, 1:25 p.m.): The response, by Massachusetts Institute of Technology atmospheric science professor Kerry Emanuel, is posted here.