解决脑机接口中脑电数据噪声问题,提升情绪识别准确率
Uncertainty-Aware Cross-Modal Knowledge Distillation with Prototype Learning for Multimodal Brain-Computer Interfaces
- 用原型学习对齐跨模态特征语义,缓解模态差异
- 引入任务专用蒸馏头,修复标签不一致导致的误差
- 在公开数据集上显著优于单模态与多模态基线方法
脑电图(EEG)是脑机接口中认知状态监测的基础模态,但易受信号固有误差和人为标注误差影响,导致标签噪声并降低模型性能。为提升EEG学习效果,已有研究探索将视觉模型中的丰富表征通过多模态知识蒸馏(KD)迁移至基于EEG的模型。然而,当前方法面临两大挑战:模态差距与软标签错位。前者源于EEG与视觉特征空间的异质性,后者则因标签不一致导致真实标签与蒸馏目标之间的偏差。本文针对由模糊特征和弱定义标签引起的语义不确定性,提出一种新型跨模态知识蒸馏框架,通过原型相似性模块对齐特征语义,并引入任务特定蒸馏头以解决监督中由标签引发的不一致性。实验表明,该方法在公开多模态数据集上显著提升了基于EEG的情绪回归与分类性能,优于单模态及多模态基线方法。结果验证了该框架在脑机接口应用中的潜力。
原文摘要 · Abstract (English)
Electroencephalography (EEG) is a fundamental modality for cognitive state monitoring in brain-computer interfaces (BCIs). However, it is highly susceptible to intrinsic signal errors and human-induced labeling errors, which lead to label noise and ultimately degrade model performance. To enhance EEG learning, multimodal knowledge distillation (KD) has been explored to transfer knowledge from visual models with rich representations to EEG-based models. Nevertheless, KD faces two key challenges: modality gap and soft label misalignment. The former arises from the heterogeneous nature of EEG and visual feature spaces, while the latter stems from label inconsistencies that create discrepancies between ground truth labels and distillation targets. This paper addresses semantic uncertainty caused by ambiguous features and weakly defined labels. We propose a novel cross-modal knowledge distillation framework that mitigates both modality and label inconsistencies. It aligns feature semantics through a prototype-based similarity module and introduces a task-specific distillation head to resolve label-induced inconsistency in supervision. Experimental results demonstrate that our approach improves EEG-based emotion regression and classification performance, outperforming both unimodal and multimodal baselines on a public multimodal dataset. These findings highlight the potential of our framework for BCI applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。