用脑电波客观评估语音增强效果,帮听障人群选更合适的助听方案。
EffortNet: A Deep Learning Framework for Objective Assessment of Speech Enhancement Technologies Using EEG-Based Alpha Oscillations
- 基于脑电α波变化,用深度学习模型量化听觉努力程度。
- 仅需40%新用户数据,分类准确率达80.9%,优于传统模型。
- 发现Transformer增强语音更接近自然语音,适合个性化听力评估。
本文提出EffortNet,一种基于脑电图(EEG)的深度学习框架,用于解码个体在语音理解过程中的听觉努力。研究采集了122名受试者在四种条件下(清晰语音、噪声语音、MMSE增强语音、Transformer增强语音)的64通道脑电数据。统计分析表明,α波(8-13 Hz)在噪声语音处理时功率显著升高,验证其作为听觉努力客观生物标志物的有效性。为应对脑电信号的个体差异,EffortNet融合自监督学习、增量学习和迁移学习三种范式。实验显示,该模型仅用新受试者40%训练数据即达80.9%分类准确率,显著优于传统CNN(62.3%)和STAnet(61.1%)。基于概率的指标显示,Transformer增强语音引发的神经反应更接近清晰语音,此结果与主观可懂度评分矛盾,但符合客观度量标准。该框架为听力技术个性化评估提供可行方案,对设计认知感知型语音增强系统具有重要意义。
原文摘要 · Abstract (English)
This paper presents EffortNet, a novel deep learning framework for decoding individual listening effort from electroencephalography (EEG) during speech comprehension. Listening effort represents a significant challenge in speech-hearing research, particularly for aging populations and those with hearing impairment. We collected 64-channel EEG data from 122 participants during speech comprehension under four conditions: clean, noisy, MMSE-enhanced, and Transformer-enhanced speech. Statistical analyses confirmed that alpha oscillations (8-13 Hz) exhibited significantly higher power during noisy speech processing compared to clean or enhanced conditions, confirming their validity as objective biomarkers of listening effort. To address the substantial inter-individual variability in EEG signals, EffortNet integrates three complementary learning paradigms: self-supervised learning to leverage unlabeled data, incremental learning for progressive adaptation to individual characteristics, and transfer learning for efficient knowledge transfer to new subjects. Our experimental results demonstrate that Effort- Net achieves 80.9% classification accuracy with only 40% training data from new subjects, significantly outperforming conventional CNN (62.3%) and STAnet (61.1%) models. The probability-based metric derived from our model revealed that Transformer-enhanced speech elicited neural responses more similar to clean speech than MMSEenhanced speech. This finding contrasted with subjective intelligibility ratings but aligned with objective metrics. The proposed framework provides a practical solution for personalized assessment of hearing technologies, with implications for designing cognitive-aware speech enhancement systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。