提升多感官反馈下错误脑电的解码稳定性
Decoding Error-Related Potentials under Multisensory Feedback with Varying Congruency
- 采用多分支EEGNet结构+辅助监督，不依赖特定感官假设
- 在异质感官条件下分类准确率稳定，多模态下优于基线模型
- 适合人机交互中复杂反馈场景的神经信号解码应用
错误相关电位（ErrPs）是人机交互中与错误处理相关的广泛研究的神经标志。在真实场景中，错误感知常伴随异质性多感官反馈，感官模态和反馈一致性差异带来的变化给可靠的ErrP解码带来挑战，尤其是不一致反馈会增加解码难度并降低分类性能。为应对这一问题，本文研究了在视觉、听觉和触觉多模态反馈下，控制感官一致性条件时的鲁棒ErrP解码学习策略。采用基于多分支EEGNet的架构并引入辅助监督，以提升在异质条件下的鲁棒性，且无需依赖显式的模态特异性假设。实验基于迷宫观察任务，设置单模态、双模态和三模态反馈配置。跨被试结果表明，所提方法在不同感官条件下表现一致，且在多模态反馈下相比基线EEGNet模型显著提升准确率。结果表明，合理的架构设计与训练策略可有效提升异质多感官条件下ErrP解码的稳定性。
原文摘要 · Abstract (English)
Error-related potentials (ErrPs) are widely studied neural signatures associated with error processing in human-machine interaction. In realistic settings, error perception often occurs under heterogeneous multisensory feedback, where variability induced by sensory modality and feedback congruency poses challenges for reliable ErrP decoding. In particular, incongruent feedback is associated with increased decoding difficulty and reduced classification performance. To address this challenge, we investigate learning strategies for robust ErrP decoding under multimodal visual, auditory, and tactile feedback with controlled sensory congruency. We adopt a multi-branch EEGNet-based architecture with auxiliary supervision to improve robustness across heterogeneous conditions, without relying on explicit modality-specific assumptions. Experiments were conducted using a maze-observation task with unimodal, bimodal, and trimodal feedback configurations. Across subjects, the proposed approach achieved consistent classification performance across heterogeneous sensory conditions and showed improved accuracy compared to baseline EEGNet models, particularly under multimodal feedback. These results suggest that appropriate architectural design and training strategies can improve the stability of ErrP decoding under heterogeneous multisensory conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。