A mixed-effects deep learning model to diagnose COVID-19 from computed tomography imaging

Severe acute respiratory syndrome coronavirus 2 (SARS‑CoV‑2) infection is usually diagnosed via reverse transcription PCR or RT-PCR, but this method is far from perfect. According to one review, RT-PCR has a real-world sensitivity of about 70% and a specificity of 95%, although negative results do not necessarily indicate a negative diagnosis. 

Additionally, a systematic analysis of individual patient data found that RT-PCR frequently misses SARS‑CoV‑2 and that early sampling may reduce false negatives. Consequently, these tests are often more helpful in ruling in COVID-19 than in ruling it out.

In cases where an RT-PCR test is negative, but a patient presents with symptoms of COVID-19, additional tests may be necessary.  

Given the exponential rise in the number of COVID-19 cases worldwide and the mounting strain on medical resources, automated diagnostic methods may effectively reduce the burden on reporting radiologists.

CT (computed tomography) images are made up of many slices, creating a three-dimensional (3D) effect. A fundamental limitation of previously reported measures using CT-scan image-based methods is the need for the same number of slices as their inputs which is not viable in different CT volumes.

Three common limitations of such models are a lack of adequately documented methods for reproducibility, failure to follow standard regulatory protocols for developing deep learning models, and an absence of external validation studies to validate the program in the real-world population.

Researchers have aimed to address the issues mentioned above in a recent study posted to the medRxiv* preprint server.

Using best practice guidelines, they developed a mixed-effects deep learning model to classify images as healthy or COVID-19 accurately. In addition, a secondary aim was to demonstrate how deep learning algorithms could satisfy current best practice guidelines for creating reproducible and less biased models.

Study details

The proposed mixed-effects model was developed using data from Russia and China retrospectively. Data in Russia were collected between March 1, 2020, and April 25, 2020. The China Consortium of Chest CT Image Investigation (CC-CCII) provided data between January 25, 2020, and March 27, 2020. The study involved data from 1,110 and 796 patients with either COVID-19 or healthy CT volumes from Russia and China, respectively.

Diagram of the overall framework. Twenty slices are chosen from a CT volume. Each slice is fed into a CNN with shared weights, which outputs a feature vector of length 2048 for each image. The feature vectors form a 20-by-2048 fixed effects matrix, X, for the GMM model with a random-effects matrix, Z, consisting of an identity matrix. A mixed-effects model is used to model the relationship between slices. Finally, a fully connected layer and sigmoid activation return a probability of the diagnosis.

Diagram of the overall framework. Twenty slices are chosen from a CT volume. Each slice is fed into a CNN with shared weights, which outputs a feature vector of length 2048 for each image. The feature vectors form a 20-by-2048 fixed effects matrix, X, for the GMM model with a random-effects matrix, Z, consisting of an identity matrix. A mixed-effects model is used to model the relationship between slices. Finally, a fully connected layer and sigmoid activation return a probability of the diagnosis.

*Important notice: medRxiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.

The proposed method used a feature extractor and a two-stage generalized linear mixed-effects model (GLMM), using back-propagation. Mixed-effects models are statistical models consisting of a fixed-effects part and a random-effects part. The fixed-effects part is used to model the relationship within the CT slice, and the random-effects part then models the spatial correlation between CT slices within the same image.

Example images showing (a) healthy and (b) COVID-19 lungs taken from the Mosmed dataset.

First, a series of CT slices forming a CT volume was fed to the model. This step was followed by a mixed-effects layer that concatenates the feature vectors into a single vector. Finally, a fully connected layer followed by a sigmoid activation gave a probability of COVID-19 for the whole volume. The mixed effects and fully connected layer with sigmoid activation were analogous to a linear GLMM in traditional statistics.

For the feature extractor, the researchers used InceptionV3, a Convoluted Neural Network (CNN). After selecting the neural network, researchers used a global average pooling layer to reduce each image to a feature vector for each slice, with a dropout of 0.6 to improve generalizability to unseen images. The feature vectors were then transformed into a 20 × 2048 matrix. The novelty of this process lay in researchers using a mixed-effects layer previously only used in hardcore statistical calculations to model the relationship between slices.

Researchers used an internal validation dataset to train the proposed model. The model showed an AUROC (area under receiver operating curve) of 0.936 (95%CI: 0.910, 0.961). Using an optimal cut-off point of 0.740, the sensitivity, specificity, Negative Predictive Value (NPV), and Positive Predictive Value (PPV) were 0.807 (0.761, 0.853), 0.953 (0.908, 0.853), 0.983 (0.966, 1.0), and 0.596 (0.513, 0.678), respectively.

On validating the model externally, it attained an AUROC of 0.930 (0.914, 0.947). Using an optimal cut-off point of 0.878, the sensitivity, specificity, NPV, and PPV were 0.758 (0.722, 0.793), 0.963 (0.939, 0.987), 0.979 (0.965, 0.993), and 0.636 (0.587, 0.685), respectively.

When 20% of the CT-scan images were missing in the validation dataset, there was a statistically significant reduction in the model’s predictive performance. Although, even at 50% data-missingness, the model continued to perform relatively well, with an AUROC of 0.890 (95% CI: 0.868, 0.912), highlighting its robustness.

Implication

This study is another indicator of leveraging the power of deep learning for screening and monitoring COVID-19 in a clinical setting. However, validation in the intended setting is vital, and models should not be adopted without this. This study also highlighted the importance of external validation in developing a robust model for disease prediction.

*Important notice: medRxiv publishes preliminary scientific reports that are not peer-reviewed and, therefore, should not be regarded as conclusive, guide clinical practice/health-related behavior, or treated as established information.

Journal reference:
Sreetama Dutt

Written by

Sreetama Dutt

Sreetama Dutt has completed her B.Tech. in Biotechnology from SRM University in Chennai, India and holds an M.Sc. in Medical Microbiology from the University of Manchester, UK. Initially decided upon building her career in laboratory-based research, medical writing and communications happened to catch her when she least expected it. Of course, nothing is a coincidence.

Citations

Please use one of the following formats to cite this article in your essay, paper or report:

  • APA

    Dutt, Sreetama. (2022, February 02). A mixed-effects deep learning model to diagnose COVID-19 from computed tomography imaging. News-Medical. Retrieved on July 23, 2024 from https://www.news-medical.net/news/20220202/A-mixed-effects-deep-learning-model-to-diagnose-COVID-19-from-computed-tomography-imaging.aspx.

  • MLA

    Dutt, Sreetama. "A mixed-effects deep learning model to diagnose COVID-19 from computed tomography imaging". News-Medical. 23 July 2024. <https://www.news-medical.net/news/20220202/A-mixed-effects-deep-learning-model-to-diagnose-COVID-19-from-computed-tomography-imaging.aspx>.

  • Chicago

    Dutt, Sreetama. "A mixed-effects deep learning model to diagnose COVID-19 from computed tomography imaging". News-Medical. https://www.news-medical.net/news/20220202/A-mixed-effects-deep-learning-model-to-diagnose-COVID-19-from-computed-tomography-imaging.aspx. (accessed July 23, 2024).

  • Harvard

    Dutt, Sreetama. 2022. A mixed-effects deep learning model to diagnose COVID-19 from computed tomography imaging. News-Medical, viewed 23 July 2024, https://www.news-medical.net/news/20220202/A-mixed-effects-deep-learning-model-to-diagnose-COVID-19-from-computed-tomography-imaging.aspx.

Comments

The opinions expressed here are the views of the writer and do not necessarily reflect the views and opinions of News Medical.
Post a new comment
Post

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.

You might also like...
Increased emotional sensitivity linked to previous COVID-19 infection, new research suggests