Idaho has its own scientific model for coronavirus. But will decision makers heed its data?
“What will the coronavirus pandemic look like a couple of weeks from now?” is a question that lies at the heart of every public health decision being made.
Giving an exact answer to that question is an impossible task, as no one can predict the future. But epidemiologists, statisticians and mathematicians all over the world — and researchers right here in Idaho — are trying to provide an answer that closely resembles what will actually happen.
When the pandemic hit the U.S. earlier this year, the Idaho governor’s office, in conjunction with the Idaho Department of Health and Welfare, created a Coronavirus Working Group tasked with designing and coordinating the state’s response to the epidemic.
The group then asked researchers all over Idaho to develop a model that the governor’s office could use to inform its health policies. The selected model was developed by a researcher working with the University of Idaho’s Institute for Modeling Collaboration and Innovation, Dr. Ben Ridenhour.
Scientific models are a good way of approximating the real outcome of an event, but they’re not easy to make. The process involves using variables that are known to affect that outcome, while also considering that those variables can affect each other and change over time.
A good model is only as good as the information used to generate its predictions. Generally speaking, models for epidemiology consider many disease-specific characteristics together with some measures of people’s behavior.
Disease-specific characteristics are relatively easy to obtain, usually from the scientific literature. They include things like how long you are going to be sick, how long you will be infectious, how likely it is that you’ll get infected if you are exposed, or how illness severity changes in different age groups.
Measures of people’s behavior are much more difficult to include in a model – those actions vary too much.
For example, when a stay-home order is implemented, some people will keep having contact with many other individuals — essential workers, for example — whereas others can remain isolated for long periods of time.
To complicate things further, some people may be more willing to comply with social distancing and security measures at the beginning of such an order but not after some time has passed.
Finally, researchers have to make sure that the model works and it approximates the data that is being observed, at least to some degree. If the model can’t accurately describe reality, then it needs to be adjusted until it does.
A COVID-19 model tailored for Idaho
Ridenhour is an assistant professor at the University of Idaho’s College of Science and has been developing models for infectious diseases since 2008, when he started modeling the flu for the Centers for Disease Control and Prevention in Atlanta.
He talked to the Idaho Statesman about the model he developed.
Idaho Statesman: How is this model different from other coronavirus models out there?
Ben Ridenhour: The basic model structure is very similar to those being used elsewhere, like the one developed by the University of Washington. The special thing about our model is that is tailored for the Idaho population and disease transmission, to help us understand what’s going on in our own state.
IS: Which are the most important parts of the model?
BR: There are a few. Probably the most important parameter is how often people contact each other, because that’s what determines how many people get infected from a newly infected person. This changes according to social distancing measures. Another important part is the transmission probability, which is the chance that you actually get the virus. Disease-specific parameters, like how long do you have the disease, are important because they determine how big the wave is. And then there are the epidemiological parameters, such as how severe is the disease depending on your age or ethnicity; we like to model that because we’re really interested in the burden on the health care system.
IS: How confident are you in the data used in the model?
BR: It depends on which type of parameter you consider. We feel pretty good about disease-specific parameters because we can estimate them using data from Idaho or from other places. We are much less confident in what I call the ‘intervention’ or the ‘public health’ parameters. For example, compliance with stay-at-home or mask-wearing orders varies from person to person — depending on people’s attitudes towards government and disease — and may change as time goes by.
IS: How do you estimate those ‘public health’ parameters?
BR: Those are really hard to know, and they require a bit of fitting the data that we had in the past and a bit of guessing as well. We need to make sure that the model is relatively close to what we actually observed in the past, that’s the model fitting part. And we also let those numbers vary as we include them in the model, which creates uncertainty in the final model estimates. But even when we fit the model as best as we can, sometimes that doesn’t help us going into the future, because people might be less likely to follow public health measures if they are in place for a long time.
IS: We have talked about the data that goes into the model. Now tell us about the information you get once you run the model calculations. What kind of data do you obtain and how do you use it?
BR: At the very basic level, the model estimates how many people are going to be sick at a certain time, how many people we think might be in the hospital at that same time, and the number of people that might die in some time frame as well. But really what we use the model for is judging what kind of effects we could expect with the various interventions that are being proposed and put in place. We vary those ‘public health’ parameters, for example, assuming that we open up schools, or that people don’t wear masks, or that we move back to Stage 2, or things like that.
IS: One of the features of your model is that it can make estimates at different spatial scales in Idaho, from small towns and cities to the entire state. Is it difficult to integrate all that information?
BR: If we don’t worry about making everything different between the cities, it’s not that hard, and the results depend on how connected are the small towns to the big towns. The harder part is that there are reasons to believe certain cities might react differently to the governor’s orders than other places. Those differences are very hard to take into account. We have another study going on that’s aimed at helping us understand that better within Idaho, but we haven’t been able to complete it.
IS: How far into the future can you make predictions with this model?
BR: The model is able to estimate these parameters for a very long time. However, with our model, the most I would trust is three or four weeks from now, especially as things are changing so rapidly now. With more time, the uncertainty in our estimates gets bigger and bigger to the point where it’s not useful anymore. We recently got a request from the state to do some modeling of scenarios for the fall and that’s too far away to really model anything we can trust. It’s just like predicting the weather: You can probably trust the forecast for maybe the next three days or something, but after that, who knows?
IS: How often do you update the model?
BR: We’re trying to update it fairly regularly. We get official data from the Department of Health and Welfare on Fridays, but we are also using the state’s Tableau data to help us do things a little bit more rapidly. And as we move between phases, we’re updating the model and trying to make sure that it’s in touch with what’s actually going on.
IS: Has the model changed since you started using it at the beginning of the year?
BR: Our understanding of some parameters has improved, like mortality rates or severity of illness for different age groups. There’s a bit of a mystery that’s going on now, which is this observed drop in mortality rate that seems to be happening, that’s something that we’re looking to update still. The other big change is in the contact patterns as we progress through the various phases of reopening. Every time that happens, we try to update the model looking at, for example, mobility patterns on Google.
IS: Has your model been well received by the governor’s Coronavirus Working Group?
BR: I was really impressed by how quickly they asked for modeling assistance. They took it to heart, and the governor immediately put a stay at home order. And immediately after that order went into place, within a week our cases were falling off, and we went down to a fairly low level of transmission. I think the harder part is now, where the government is still asking for input from the model, but there are political and economic considerations that are outside of the reach of the model. All we can do is provide them with the best information about what we expect from the public health and the disease transmission side of things.
IS: What is the model predicting for the pandemic in Idaho in the coming weeks?
BR: According to the model, we expect to see increases in disease going forward for some time, and at a pretty steep rate, because we’re nowhere near the herd immunity level that we would need to slow down the disease. If we’re going to put interventions in place, now’s the time to do it. There are plenty of studies that show that for every day you wait, there are more and more consequences for the size of the pandemic.
IS: Why is this model not currently available for the general public?
BR: Part of it is because the data are protected by HIPAA rules, and we want to make sure that we aren’t violating any privacy concerns. And the other part is that we want to develop a nice interface and that has been slow. But we hope to get something out there in the very near future.
