Pandemic Makes Modeling’s Importance Clear

As the trajectory of the COVID-19 pandemic widened and sped up in February and March 2020, overtaking the Americas as it had already stricken Asia and Europe, much of the estimation of how the disease might spread, especially in the U.S., was based on a single model created by the Institute for Health Metrics and Evaluation (IHME) at the University of Washington.

Policy makers at all levels of government watched the IHME's range of projections about the possible number of hospital beds and ventilators that would be needed and the number of people who would die and acted based in part on what they saw. National news organizations used the model as a foundation for stories about what the public would need to do, whether mandated by law or not, to slow the spread of the virus.

Within a matter of days, however, the IHME had plenty of company from models created by researchers utilizing tried and true methodologies such as SEIR (Susceptible, Exposed, Infected, Recovered) modeling, as well as incorporating new elements such as GPS mobile device data and electronic health record data. To name just a few:

• A team from the Massachusetts Institute of Technology used partnerships with Hartford, CT-based Hartford Healthcare and the Italian health system ASST Cremona to create a model of projected health system needs such as ICU beds and ventilators, and deaths on state and national levels in the U.S. (Italian data fed the model for possible case trajectories in the U.S., as well as Italy.)
• Researchers at the University of Virginia (UVA) Biocomplexity Institute created a model for the commonwealth's public health officials that measured the likely impact of interventions such as shelter-in-place orders, and the estimated results of COVID caseloads under different scenarios of lifting restrictions.
• Researchers at the University of Texas at Austin led by mathematical epidemiologist Lauren Ancel Meyers created the UT COVID-19 Modeling Consortium, an interdisciplinary network of researchers and health professionals building models to detect, project, and combat COVID-19.
• The University of Pennsylvania's Wharton School of Business released a coronavirus-specific version of its Penn Wharton Budget Model on May 1.

In fact, the number of COVID-19 models became so numerous that one lab, the U.S. Centers for Disease Control (CDC) Influenza Forecasting Center of Excellence at the University of Massachusetts, created a COVID-19 forecasting hub, with an ensemble output of numerous models.

"None of these models on their own are adequate to drive policy decisions," center director Nicholas Reich said in a statement announcing the hub's debut. "We've created a simple ensemble model to try to unify all these forecasts of COVID-19 together. Some models are overly optimistic, and others may be overly pessimistic. The reality is likely in the middle. We need the diversity of these modeling teams to understand the full range of future possibilities."

Complex Systems Approach Needed

The speed and severity with which the virus spread led not just to the fast development and publication of models from all corners of the U.S. and the globe, but also a belief that epidemic modeling's role would require a much more comprehensive analysis of contemporary society's economic, political, and social dynamics.

Meyers, who said she has spent most of her career modeling disease, said it was too early to try to describe accurately just what changes the discipline will undergo, but she also said she had noticed profound changes in just a few short weeks.

"It's so hard, being in the thick of it, to know how this will change our field, our way of modeling, but it certainly already has and will," Meyers said. "The pace of innovation is unprecedented, for many, many different reasons. We are getting more direct requests to build different kinds of models that can do things that were never asked of our models before."

So numerous and varied were the requests coming in that Meyers, who said she was running "a little research group of five to seven people" prior to the outbreak of the virus, launched the consortium, which combines the talents of her team with those of social scientists and engineers.

"Every morning at 9:30 we have a Zoom call, and there are usually dozens of people on, from the University of Texas and from all over the country and the world, including people from other fields. We're able to pull in different perspectives and different techniques that had never really appeared on our radar."

Madhav Marathe, director of the network systems science and advanced computing (NSSAC) division of the UVA's Biocomplexity Institute, said he thinks this multi-disciplinary collaboration might be a lasting legacy of the complexity of modeling COVID-19.

"It's become much more apparent to the world now that epidemic science cannot be studied as just one of transmitting diseases on social networks," Marathe said. "The economic and social and political impact on this pandemic and the reverse – the impact of the pandemic on these systems – is at least as big as the pandemic itself. So I think when people study epidemics in the future, they will have to take into account this complex systems approach to this."

Small devices, big data

Perhaps the most vital pool of data available thus far to modelers in this pandemic has been aggregated and anonymized geo-location data from mobile phones. For the first time, this data has allowed modelers not only to see where people have traveled during early days of the outbreak, but also the likely effects of what might happen when widespread social isolation orders are rescinded. This data has also taken on extra emphasis in the lack of widespread testing in many locales.

"Having worked in the field of pandemic modeling for most of my career, we have learned lots and lots of scenarios," said Meyers, who used SafeGraph mobility data for one of the consortium's first studies. "We have developed lots of tools and derived optimal strategies. And I don't think ever once did we have a scenario where the only viable intervention to prevent overwhelming surges in hospitalizations and deaths was to completely shelter in place, to completely lock down the population.

"We've never been in a situation where that data is so vital, because we've never shut down society before. So even though that data has existed, there's never been a need for it, or such rapid innovation around how to use it."

The UVA team also used aggregated mobility data (from Google and Cubeiq) in their model. In the short term, Marathe said, the data offered one of the few quickly available metrics on COVID-19's possible spread: the UVA model, for example, estimated that only 15% of actual cases were diagnosed, due to the small number of people who had been tested for the disease.

"We are in this supposedly big data world, but if you really drill down, so little data is available to do this model," Marathe said. "One of the biggest missing pieces is a sense of how many people are really infected."

What the team was able to do, however, was to associate Virginia's social distancing mandate, which went into effect March 15, with the aggregated mobility data and a resultant decrease in the growth of the disease: at the time the team presented its model to state government officials, the Google data, for example, noted a 44% decrease in the percentage of visits to retail and recreation service establishments, and a 39% reduction in travel to workplaces. Cubeiq data estimated a greater than 50% reduction in average individual mobility from the annual average. The disease reproductive number rate, or R0, according to state health department data, was halved, from 2.2 to 1.1; not quite a trajectory that would result in ending the pandemic, but certainly one that would slow demand on health systems."

Health systems, modelers collaborate

The MIT model, named DELPHI ((Differential Equations Leads to Predictions of Hospitalizations and Infections), employed data from existing studies as well as updates from its health system partners in Europe and the U.S. to create its projections.

Project leader Dimitris Bertsimas, the associate dean of business analytics at MIT's Sloan School of Management, said the team was starting to employ electronic health records data from the partnering health systems in Italy and Hartford for mortality and intensive care unit (ICU) projections, but that some of the process of data integration was not automated. Bertsimas said it took about two weeks to integrate EHR data from the health systems into the model.

The MIT team also added a component modeling projected ventilator need, both between component hospitals in its partners' systems, for which data was readily available, as well as more uncertain numbers of ventilators available on a state-by-state basis nationally. As some states saw a surge in severe cases early on, and federal response was clouded by a lack of information and transparency regarding how many ventilators were available and their locations, this model helped recommend a way to allocate equipment between states.

Bertsimas said that despite the dearth of real-time data on the federal stockpile, previously published data allowed his team to model an adequate number of ventilators nationwide, with shortages in "hot spots" emerging at different times. The model estimated which shortages could be mitigated by shipping equipment from areas with surpluses. That several states did indeed share ventilators with others in need of more did not surprise Bertsimas, but he did not speculate on how great a role the DELPHI data played in those transactions.

"This is also a politically charged story, as you know," he said. "Our view was not to say 'Governors, please use our model,' but basically to make the point of looking at when peaks happen, and therefore add us to what you need."

Efficiency begets fragility

For Chris Barrett, the UVA Biocomplexity Institute's executive director, the COVID-19 pandemic presents a profound illustration of how the march of technology and information that supported the evolution of a real-time global economy can also be exposed as fragile enough that society can grind to a virtual standstill.

"One of the things I've seen in this is the depth of societal integration," Barrett said. "It's something everybody talks about, but to see it turned against you in a contagion scenario is something quite remarkable. The basic motivations for distribution of labor and labor specialization, the depth that has gotten to because of our ability to connect financially and through transport and information, is something that bears a really hard look.

"Everything is fantastic and works better than it could possibly work as long as everything works. But as soon as something doesn't work, it just bangs around inside these multi-networks and creates damage in places that a small amount of oversupply, that form of inefficiency, would be valuable."

One of those beneficial inefficiencies may be the ongoing maintenance of the modeling community's work in prior epidemics by public health officials, as well as policy makers. For example, Meyers said Texas state health department officials contacted her group after the 2009 H1N1 flu epidemic asking for tools to help them address real-time challenges, such as which pharmacies were best located to serve underinsured populations, and ventilator distribution.

"We built those models, engaged with the state for several years, and then the models just sat on the shelf," Meyers said. "The people engaging with us got moved to other positions. There was a lack of sustained investment and interest. So even though those tools are there, they are not ready to go.

"One of the things we can do is ramp those things back up, and let public health departments and policy makers know those tools are there. In fact, one of the conversations I had this week was explaining to state leaders that tool is there and has the zip code of every pharmacy in the state. Even though we don't have anti-virals to deploy yet, we do need to figure out how we are going to set up testing. That tool could help make that process much more efficient."

For UVA's Barrett, tools like that exemplify the need to emphasize the interconnectedness of all the data of modern life — of individual travel patterns and commodity flows, of labor policy, of government transparency and sense of global responsibility, or what might happen if that sense is neglected — will require both a new level of sophistication about the dynamics of modern society, and sophisticated analytic machinery to help drive decisions.

"To understand these principles, if we don't have formal abstractions, if we don't have computational networks that scale, I don't think we can understand this at the extent to which it has happened. The depth of societal integration is such that there is no way to understand this without those kinds of tools. It's not possible."

Gregory Goth is an Oakville, CT-based writer who specializes in science and technology.