“Both policy makers and voters need to know that we still cannot answer most basic questions about this pandemic with the tools we have on hand,” says Douglass, who has been a vocal Twitter critic of the type of studies like the one published this week about Sturgis and has coauthored a forthcoming paper on similar shortcomings in the rapidly ballooning Covid literature. And he’s not alone. Other scientists also worry that the rush to use bad or incomplete data to provide answers, any answers, on the effects of large gatherings, will ultimately do little to bring an end to the pandemic. And in fact, they think, it may do more harm than having no answers at all.
“Look, these are really pressing questions we need to sort out,” says Asish Jha, a physician, health researcher, and recently-appointed dean of the Brown University School of Public Health. “We need to be able to carefully dissect how much are protests and rallies and other events contributing to disease outbreaks. But it’s more important to get an analysis done right then to get it done fast. And in this case, it’s just not done well enough. I wish the authors, knowing how incredibly politically volatile the environment is, had done more careful work.”
Indeed, South Dakota officials quickly rejected the researcher’s findings. A spokesman for the state’s department of health told WIRED in an email that the study “does not align with the impact we’ve seen on the ground.” Governor Kristi Noem, a Republican who has eschewed mask mandates and other disease prevention measures, went farther, telling Fox News that the researchers “did some back-of-the-napkin math, made up some numbers and published them.”
Now, that’s an alternative universe of its own. The well-accepted methods employed by the San Diego State University team are standard for economists to use when trying to answer questions about the societal impacts of a sudden event. But that doesn’t mean that they were the right methods for answering this question, nor that the study is without its flaws. In particular, Jha takes issue with a lack of so-called “falsification analyses.” These tests are a way to check your work, to make sure it is measuring what you think it is. One way to do that is to imagine a world in which the rally happened three weeks earlier than it actually did, and run the models again. If the rally (the actual rally) really was the thing driving coronavirus spikes, an analysis using this fictive Sturgis date shouldn’t turn up any changes in case rates—because in this analysis the real rally is still three weeks in the future. If you do see a change, then you know something other than the Sturgis rally is causing the spike. Without those kinds of checks, it’s harder to draw claims of causation.
But the bigger issue, Jha says, is that the real-world data just doesn’t seem to line up with the study’s estimates—except in South Dakota. The state’s health department has so far identified 124 people who attended the rally and later tested positive for Covid-19. A spokesperson for the department declined to say how many close contacts of those individuals are currently being monitored to see if they become ill. But in the month following the rally, South Dakota’s daily average for new diagnoses more than tripled from 82 to 307. That’s a visible spike that lines up pretty well, timing-wise, with Sturgis, says Jha.
Based on cell phone, data though, 90 percent of the rally-goers came from places outside the state. Yet when you look at the counties identified by the Sturgis study authors as contributing the highest number of attendees, you don’t see similar surges. According to their model, in the three weeks following the rally, Covid-19 infections rose the most in places like Maricopa County, Arizona; Hennepin County, Minnesota; and three counties surrounding the Denver metropolitan area in Colorado. In those counties, the authors found the Sturgis rally to be associated with a 13.5 percent increase in Covid-19 cases. But according to state public health department data, Maricopa County actually saw a decrease in daily reported cases—from a 7-day average of 775 at the start of the rally to 266 a month later. In two out of the three Colorado counties, the number of new cases declined or went flat following the rally. That trend was also observed in some other high-flow counties, including in San Diego, Los Angeles, and Clark County, home to Las Vegas.
