By turning snore sounds into time-frequency images, researchers built a lightweight AI pipeline that may help locate upper-airway obstruction more precisely, though real-world clinical testing remains the next hurdle.
Study: Snoring classification with deep time-frequency features. Image Credit: Kleber Cordeiro / Shutterstock
In a recent 'Article in Press' in the journal Scientific Reports, researchers proposed a heterogeneous integration framework for snore-source classification.
Snoring is a primary symptom associated with obstructive sleep apnea, caused by the obstruction or vibration of the upper airway structures, including the epiglottis, tongue base, lateral oropharyngeal walls, and soft palate. The anatomical origin of snoring can be non-invasively identified by classifying snoring audio signals. However, current classification methods struggle with limited data, poor integration of time-frequency information, and imbalanced class distributions.
About the study
In the present study, researchers proposed a heterogeneous integration framework for snore-source classification. Their framework encompasses three core modules: short-time Fourier transform (STFT)-based spectrogram generation, pretrained convolutional neural network (CNN) feature extraction, and support vector machine (SVM) classification. Accordingly, STFT generates spectrograms by converting snore audio signals, retaining time-frequency information.
Furthermore, high-level time-frequency features are extracted from spectrograms using pretrained CNNs. Finally, an SVM classifier is trained on extracted features to classify snore audio signals into four categories. The team tested their model on the Munich-Passau Snore Sound Corpus (MPSSC). The corpus bundled snore recordings from the soft palate, tongue base, epiglottis, and lateral oropharyngeal walls, classified as V, T, E, and O, respectively.
The team split the MPSSC dataset into training, development, and test sets. Audio samples of these snoring classes were unevenly distributed in the training set, with class V samples accounting for 56.9% and class E samples for 10.7%. Therefore, an upsampling method was adopted to make sample counts more uniform. Next, the team applied STFT with a 512-sample window at a sampling rate of 44.1 kHz to generate spectrograms.
The spectrograms were resized to meet the input requirements of two pretrained CNNs, VGG19 and AlexNet, and 4096-dimensional features were extracted from the fully connected layers 6 (fc6) and 7 (fc7). Further, an L2-regularized SVM was trained on the extracted features. AlexNet fc7 with Viridis color mapping was found to be the best-performing combination, yielding an unweighted average recall (UAR) of 46.0% and 67.1% on the development and test sets, respectively.
To evaluate the contribution of each component in the framework, ablation analyses were performed by modifying or excluding individual modules while keeping other conditions unchanged. STFT removal and spectrogram substitution with waveform-based image representations reduced the UAR to 54.3%, a 12.8 percentage-point drop, indicating that explicit time-frequency information is crucial for feature extraction.
Further, there was a 7.5 percentage-point drop in performance when the SVM was replaced with a fine-tuned fully connected layer, highlighting the importance of the classifier choice. Notably, replacing the pretrained CNN with handcrafted features decreased UAR by 21.3 percentage points. Next, the team assessed the proposed framework's efficacy against various conventional methods.
These included Mel frequency cepstral coefficients with SVM (MFCC + SVM), end-to-end CNN, CNN-long short-term memory (LSTM) baseline, dual convolutional gated recurrent unit (DualConvGRU), audio spectrogram transformer (AST), WavLM, and wav2vec 2.0. These methods were evaluated on the same training, development, and test sets of the MPSSC.
Most methods showed higher UAR on the test set than on the development set, with the proposed framework achieving the largest improvement (21.1 percentage points). Moreover, the proposed framework outperformed MFCC+SVM, indicating that handcrafted acoustic features fail to capture complex patterns in snore audio signals. It also reported a slightly higher test-set UAR than the end-to-end CNN.
DualConvGRU had a higher UAR on the development set than the proposed framework, but its improvement from the development set to the test set was only 8.8 percentage points. The proposed model also reported higher test-set UARs than advanced audio models, such as WavLM, AST, and wav2vec 2.0. Finally, in a confusion matrix analysis, the DualConvGRU model showed marked confusion between the V and O classes, whereas the proposed framework achieved a more balanced recall profile, although O remained challenging, and T recall decreased.
Conclusions
In summary, the study described a snore classification model based on STFT spectrograms, pretrained CNN feature extraction, and SVM classification; it achieved a UAR of 67.1% on the MPSSC test set, the highest reported value among the compared methods. Removing any single module decreased UAR by 7.5 to 21.3 percentage points, underscoring the complementary roles of the modules. Further studies are needed to independently validate this model on external clinical datasets and improve its generalizability and robustness.
Study links genes and sleep habits to Alzheimer's risk