In a recent study published in Scientific Reports, researchers developed and trained an artificial intelligence (AI) deep learning model to predict the number of COVID-19 cases 14 days into the future.
Study: A novel bidirectional LSTM deep learning approach for COVID-19 forecasting. Image Credit: PopTika/Shutterstock.com
Background
This model uses a combination of daily confirmed cases, region-specific government policy, reproduction numbers, and flight details from the previous 30 days to accurately predict future COVID-19 outbreaks.
Model validation using COVID-19 data from 190 countries reveals that the model has error rates as low as 33%, improving accuracy for countries with multiple COVID-19 waves.
Deep learning models such as this may help safeguard us from future pandemics by providing policymakers with the best information to utilize their available resources.
Predictive models in pandemic forecasting
The ongoing coronavirus disease 2019 (COVID-19) pandemic remains the worst in recent history, with the World Health Organization (WHO) estimating over 771 million cases and almost 7 million mortalities thus far.
Monitoring and predicting the spread of pandemics is integral to efficient containment planning and resource allocation. While short-term predictions using time series analysis have proven useful, they do not provide policymakers with sufficient time to prevent calamities before they occur or adequately prepare in the face of unprecedented medical infrastructure requirements.
A number of research groups attempted to simulate the spread of COVID-19 during the early phases of the pandemic. The most popular approach was to use compartmental epidemiology models (e.g., SIR and SER) to identify potential hotspots for the disease.
Additionally, reproduction number (Rt) computations, the estimate of cases stemming from a single infected individual, were used to improve these epidemiological models' predictive power and accuracy.
Tapping into the computational power available to humanity today and the immense data available to train them, machine learning (ML) and deep learning models based on time series estimations were developed to predict COVID-19 outbreaks days or weeks before.
These gold standards were the Autoregressive Integrated Moving Average (ARIMA) method. However, recurrent neural networks (RNN) and their derivative long short-term memory (LSTM) were also tested by China, the United States (US), India, Canada, Australia, and some European nations.
A notable demerit of these models was that they were designed to predict outbreaks in one of a few regions/countries, preventing their use on a global scale. Furthermore, external factors, including containment policy, were not considered during their development, resulting in high error rates and poor predictive power.
About the study
The present study borrows from the US Centers for Disease Control and Prevention's (CDC's) ensembled forecasting method, which operates on the belief that mortality and prevalence of a pandemic subsides after 30 days of containment policy implementation.
In this study, researchers developed and trained deep-learning LSTM models combining multiple time-dependent factors (daily confirmed cases, Rt, containment policy, mobility, and flight data) to predict COVID-19 outcomes 14 days into the future using data from the preceding 30 days.
The training dataset comprised prevalence data from 22 January 2020 to 31 January 2021 from the Johns Hopkins University. Data for 24 time-dependent variables from 190 countries was acquired from ourworldindata.com and similar online open-source databases.
The Official Airline Guide (OAG) was used as the repository for flight data. Effective Rt was derived from Medina-Ortiz et al.'s 2020 publication on coronavirus disease Rt.
The preliminary LSTM model was feature-engineered to use the preceding 30 days of data as sequential inputs and a single prediction 14 days into the future as an output. Modeling was carried out individually for the 190 countries in training and validation.
To improve overall model accuracy and overcome LSTM's main limitation – that the current state can only be computed via the backward context, Bidirectional Long-short Term Memory models (BiLSTM) were generated and trained on the same dataset as the preliminary model.
"The BiLSTM algorithm fuses the ideal functions of bidirectional RNN and LSTM. This is done by combining two hidden states, which allow information to come from the backward layer and the forward layer."
Model hyper-parameter tuning was performed via trial and error using a rmsprop optimizer with mean absolute error (MAE) as the loss function. Model accuracy was evaluated by comparing model output with real-world data.
The statistical evaluation metrics used included Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), Mean Absolute Error (MAE), and total absolute percentage error.
Finally, model performance was compared to ARIMA model computations over the same period to establish the utility of the BiLSTM model versus the current gold standard.
Study findings
This study presents the first effort wherein multi-variable open-source data, including flight data, were leveraged to develop and train an ML model for COVID-19 outbreak predictions.
Results reveal that the model could accurately predict daily COVID-19 prevalence between 9 January 2021 and 31 January 2021 with a median error of only 35%. Maximum error readings were significantly lower than those produced by the ARIMA model, the current gold standard in pandemic prediction.
The BiLSTM algorithm additionally has the potential for further improvements to its predictive power by incorporating additional variables over the preexisting 24 and supplementary prevalence training data.
Validation using data from 84 countries revealed that the BiLSTM models performed best for countries with multiple COVID-19 waves, suggesting improved accuracy given larger training datasets.
A second model using fewer variables achieved similar accuracy and error rates, suggesting that the model remains robust even under data-deficit conditions.
"Forecasts can provide potentially useful information to facilitate better allocation of resources and containment planning by healthcare providers and help policymakers manage the consequences of COVID-19 over a longer time horizon. For future work, an ensembling approach to combine both models and potentially other time-series candidate models can be explored."
Conclusions
In the present study, researchers developed, trained, and validated a deep-learning AI model to predict COVID-19 incidences. The model used an ensemble of 24 variables from 190 countries over 30 days to forecast COVID-19 outbreaks 14 days into the future.
Model testing revealed a median error rate of 35%, which improved for countries that experienced multiple COVID-19 waves and over 10,000 confirmed cases. Maximum error rates were substantially lower than those produced by the ARIMA method, the current gold standard for pandemic forecasts.
Together, these results reveal this BiLSTM model as a robust tool to equip policymakers with the necessary information to allocate their resources best.
Did California's pediatric COVID-19 vaccination program reduce reported cases and hospitalizations?