The entire world has experienced the COVID-19 pandemic, and within three months of the first recorded case, more than half of the global population was undergoing some form of quarantine. Scientists struggled to make sense of a new, poorly understood disease, and decision-makers scrambled to find data that could help them guide policy. From roughly January 2020 to the present, scientific papers devoted to COVID-19 doubled every fourteen days, for a total of more than 100,000 papers. As the pressure for information increased, the health sciences embraced preprint publication—work that was uploaded to the web without scrutiny.
The global media has been galvanized by the COVID-19 pandemic, with print, video, and audio outlets scrambling for news. This followed a decade-long change in the media landscape. Once dominated by a small number of high-profile outlets, it has become fractured: sound bites, headlines, and video fragments now a part of the public conversation. The new media landscape did not scrupulously distinguish among peer-reviewed papers, preprint uploads, and opinion pieces. A preprint study on COVID-19 seroprevalence in Santa Clara County, California, quickly hit the front pages following its publication on the preprint site medRxiv.1 News outlets reported that the virus had spread “50 to 85 times more than confirmed cases”2 before epidemiologists had the chance to comment on the paper’s many flaws.3 Other stories promoted drugs such as hydroxychloroquine,4 and still others were devoted to predictions from various infectious disease models.5 “2.2 Million People Could Die in US,” claimed one news site, citing a controversial model released by the Imperial College in London.6 Politicians reacted to the rapidly evolving narratives, using fragmentary stories of complicated scientific observations to inform policies that ultimately influenced the lives of millions.7
All of this should be reason to suggest a reflective pause, a consideration about how science has operated over the past few months and whether the scientific community should, or could, have responded better. In many ways, science operated in the past few months exactly as it should have. The scientific community saw an important challenge and turned its energy and attention to meeting it. From another perspective, it is not at all clear that the traditional way in which science has operated has served the world well during this time.
Understanding nature is difficult business, and science is as often wrong as it is right. Linus Pauling predicted a triple helix as the foundational architecture of DNA.8 Clearly and completely wrong, his paper has been cited thousands of times. There is nothing shameful about being wrong. Peer review is designed to catch mistakes, but reviewers often see the world through the same lens as the authors they are reviewing, and it requires still other groups to get at the truth. None of this has been possible during the past few months. What then does this moment teach us about science, and the role of science in such a moment? Let me offer three observations:
First, science is complex, and it must involve many scientists working through a lengthy series of steps. Any given scientific finding is only part of the story, and hardly ever all of it. Meaningful policy can seldom hinge on a single scientific paper. This is poorly, if at all, understood outside the scientific community, even if it is essentially taken for granted within it. It falls on us to note that our work remains just one piece of a puzzle. The media will not make this case.
Second, the task of understanding nature rests on the creation of counterfactuals that do not directly reflect the messy, complex, and all-too-real world. We study motion and acceleration in a vacuum, just as we study infectious disease transmission under particular conditions of disease spread. Nearly all the predominant models of infectious disease transmission that dominated the early part of the COVID-19 conversation rested on assumptions of homogeneous mixing, that is, that people carrying the disease would mix equally with all parts of the overall population.9 But human mixing is far from homogeneous. The assumption may be perfectly valid for certain purposes. The same assumption can result in order-of-magnitude inaccuracies in infectious disease prediction when the context is actually heterogenous mixing. It is a step too far to expect that the media will report our findings with our assumptions front and center unless we double down on the effort to do just that.
Third, this moment teaches us that the world does, indeed, pay attention to science when it matters. This means being careful about what we say and write, being judicious about what makes it into the world, particularly when preprints are circumventing peer review. This requires discipline and judgment about what we publish. It requires us to assume ever greater responsibility for our work. It is simply not good enough to assume that the media will sort through our work and put it in context. That falls to us. Much has been written about the more than 16,000 deaths from COVID-19 in Lombardy, Italy.10 But why have we written so much less about the fewer than 300 deaths in Sicily?11 These two data points are each critical to understanding the COVID-19 pandemic, but a focus only on the former fuels fear and heightens concern without illuminating the complex forces—including chance, much as we shy from admitting it—that ultimately shape what we are experiencing.
COVID-19 taught us that science can rise to the occasion. That stands science in good stead. It has also taught us that science has a large role to play in informing decision making, even quick decision making. But in order for science to maintain its place of influence, it is on us, the scientists, to discharge our power responsibly. That will require us to do better than we have done over the past few months.
