New machine learning model predicts cardiac arrest in ICU patients using ECG data with high accuracy

By Lily Ramsey, LLMNov 27 2023Reviewed by Lily Ramsey, LLM

In a recent article published in Npj Digital Medicine, researchers utilized electrocardiogram (ECG) data from a large retrospective cohort to extract various heart rate variability (HRV) measures.

This data was used to develop a machine learning (ML)-based real-time predictive model for sudden cardiac arrest predictions in critical care settings.

The study validated the model on registry data of intensive care unit (ICU) patients at Seoul National University Hospital (SNUH) in South Korea between March 2020 and August 2022. They excluded data from patients under 18 and those without ECG data.

Study: Real-time machine learning model to predict in-hospital cardiac arrest using heart rate variability in ICU. Image Credit: Roman023_photography/Shutterstock.com

Background

There have been advancements in critical care medicine; however, the incidence of unexpected and sudden cardiac arrests remains high in patients admitted to ICUs.

Reasons for in-hospital cardiac arrest are diverse; thus, developing an accurate prediction model for real-time detection of cardiac arrest in ICU settings and their rapid treatment, including cardiopulmonary resuscitation (CPR) and early defibrillation, is critical for improved patient outcomes.

Earlier models used clinical features extracted from electronic medical records (EMRs) for in-hospital cardiac arrest predictions, which displayed good discriminatory performance. However, the need to collect various EMR variables limits these models.

On the other hand, ML algorithms-based ECG-based prediction models can ensure constant, real-time detection and monitoring of critically ill patients, especially for in-hospital cardiac arrest.

So far, researchers have tested several ECG-based markers, such as Q wave, R wave, and S wave (QRS complex) prolongation, heart rate, and heart rate variability (HRV) as predictors in their ML models.

From these, HRV, a measure of time variance between consecutive heartbeats (RR intervals), has emerged as the most reliable predictor of cardiac arrest.

There are several other HRV measures, such as the standard deviation of normal RR intervals (SDNN) and HRV triangular index (HTI), to name a few, and all arise from a single ECG source. 

While the conventional logistic regression models overlook the diverse information offered by multiple HRV measures, the new-generation ML models can learn about intricate relationships among several HRV variables.

Several studies have highlighted the effectiveness of ML algorithms using HRV measures to predict sudden cardiac arrest; however, studies using several HRV measures (simultaneously) from only ECGs are scarce.

Moreover, studies often lack a large sample population of ICU patients to extract ECG data.

About the study

In the present study, researchers constructed a structured ECG dataset of 5,679 ICU stays from 4,821 patients, with (event group) and without (control group) sudden cardiac arrests from 0.5 to 24 hours before the event, with five-minute epochs at five-minute intervals.

They used epochs from ECG signals to calculate HRV features and predict the event.

Then, they divided the dataset into development (80%) and validation (20%) sets at the patient level. There were 634,396 and 139,663 epochs in the development and validation sets, respectively, with event rates of sudden cardiac arrest of 1.24% and 1.35%.

The Neurokit2 Python library provided 74 HRV measures, including time-domain, frequency-domain, and nonlinear measures, of which the BorutaShap algorithm selected 33 as the input features of the prediction model.

Utilizing such a comprehensive set of HRV measures, they developed a light gradient boosting machine (LGBM) algorithm that increased the accuracy and performance of the prediction model. 

Further, the researchers optimized the hyperparameters of the LGBM model using Bayesian optimization to find the best model training parameters, which they tested on the validation set. 

The primary outcome was the occurrence of cardiac arrest within 0.5–24 hours. The secondary outcomes included the occurrences of cardiac arrest from 0.5 to 18, 12, 6, 3, and 1h. 

The team evaluated the discrimination performances of the model using the area under the receiver operating curve (AUROC), the area under the precision-recall curve (AUPRC), and the 95% confidence interval (CI).

Additional metrics for assessing model performance were specificity, sensitivity, precision, accuracy, and F1-score. 

Furthermore, they employed the Shapley additive explanations (SHAP) method for feature importance analysis, which explained the output of the ML model based on a game-theoretic framework.

The SHAP assigned a unique contribution value to each feature of the model based on its impact on the prediction outcome.

Results

For the primary outcome, the model achieved an AUROC and AUPRC of 0.881 and 0.104 [95% confidence interval (CI)].

While the AUROC of the secondary outcomes was comparable to that of the primary outcome, the AUPRC decreased as the prediction period narrowed from 6 to 1 hour and neared sudden cardiac arrest; consequently, the model increasingly overpredicted event. 

Moreover, the model exhibited consistent and reliable calibration for secondary outcomes at 18 and 12h. The AUROCs of subgroup analyses did not show substantial variations between the two patient monitors.

Furthermore, compared to a clinical parameter-based model, which utilized 43 features derived from six vital signs, the study model using HRV measures showed a significantly higher AUROC (0.881 vs. 0.735, p < 0.001).

The feature importance analysis results also revealed the top six features in the study model.

These were the 20th percentile of the RR intervals (Prc20NN), triangular interpolation of the RR interval histogram (TINN), and the interquartile range of the RR intervals (IQRNN), values for which began increasing at nearly six hours before a sudden cardiac arrest, after which dynamic changes occurred.

The HTI values increased until cardiac arrest, whereas the acceleration/deceleration segments (IALS) and the minimum of RR intervals (MinNN) values decreased by that time.

TINN and HTI are both time-domain HRV measures, where a larger TINN value signifies greater variability in the RR intervals, and lower and higher HTI values suggest a higher proportion of intervals cluster around the mode and wider intervals spread, respectively.

A prior study revealed that IALS was significantly higher (mean value =0.78) in patients with congestive heart failure (CHF).

Similarly, the current study results suggested that higher IALS can be associated with compromised cardiac conditions, and nearly 30–50% of the patients with CHF were at risk of sudden cardiac arrest.

Conclusions

The proposed study model is highly accessible and transferable to other healthcare settings that collect ECG data. It offers easy application in clinical practice because continuous ECG monitoring is a standard practice in ICU settings.

Previous studies have not documented consecutive changes in HRV measures. Therefore, this study's analytical results could provide valuable insights into the real-time evaluation of a patient and facilitate the prompt initiation of interventions to prevent sudden cardiac arrest.

However, the causality between these HRV measures and cardiac arrest requires further investigation.

To conclude, if future studies validate these results, they could help detect in-hospital cardiac arrest in critically ill ICU patients.