研究非侵入性脑信号语音解码的鲁棒性,发现归一化方式比模型复杂度更重要。
Measuring Robustness of Speech Recognition from MEG Signals Under Distribution Shift
- 通过改进归一化、分组和标签平衡提升解码性能
- 最佳模型在测试集达60.95% F1-macro,显著优于基线39.53%
- 强调处理分布偏移对单次试验解码的关键意义
本研究基于2025 PNPL竞赛的LibriBrain音素分类基准,探究从非侵入性脑磁图(MEG)信号中进行语音相关解码的鲁棒性。对比了残差卷积神经网络(CNN)、基于STFT的CNN及CNN-Transformer混合模型,并考察了分组平均、标签平衡、重复分组、归一化策略与数据增强的影响。实验表明,预处理与数据配置选择比模型架构复杂度更具影响,其中实例归一化是提升泛化能力最有效的修改。自研最强模型(结合分组平均、标签平衡、重复分组与实例归一化)在测试集上取得60.95% F1-macro,远超基础CNN的39.53%。然而,多数未使用实例归一化的模型在验证集到测试集间出现显著性能下降,表明由归一化统计量差异引发的分布偏移是主要障碍。相比之下,MEGConformer在验证集与测试集均保持64.09% F1-macro,且显著性图分析显示:弱模型在各数据分裂中呈现更集中或重复的音素敏感模式,而MEGConformer则表现出更分散的特征响应。结果表明,提升非侵入性音素解码可靠性需更好应对归一化相关的分布偏移,同时解决单次试验解码挑战。
原文摘要 · Abstract (English)
This study investigates robust speech-related decoding from non-invasive MEG signals using the LibriBrain phoneme-classification benchmark from the 2025 PNPL competition. We compare residual convolutional neural networks (CNNs), an STFT-based CNN, and a CNN--Transformer hybrid, while also examining the effects of group averaging, label balancing, repeated grouping, normalization strategies, and data augmentation. Across our in-house implementations, preprocessing and data-configuration choices matter more than additional architectural complexity, among which instance normalization emerges as the most influential modification for generalization. The strongest of our own models, a CNN with group averaging, label balancing, repeated grouping, and instance normalization, achieves 60.95% F1-macro on the test split, compared with 39.53% for the plain CNN baseline. However, most of our models, without instance normalization, show substantial validation-to-test degradation, indicating that distribution shift induced by different normalization statistics is a major obstacle to generalization in our experiments. By contrast, MEGConformer maintains 64.09% F1-macro on both validation and test, and saliency-map analysis is qualitatively consistent with this contrast: weaker models exhibit more concentrated or repetitive phoneme-sensitive patterns across splits, whereas MEGConformer appears more distributed. Overall, the results suggest that improving the reliability of non-invasive phoneme decoding will likely require better handling of normalization-related distribution shift while also addressing the challenge of single-trial decoding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。