提出新型脑电注意力解码模型,更好利用频段信息提升听觉关注识别准确率。
FAConformer: Frequency-Aware Convolutional Transformer for Auditory Attention Decoding

- 分频段独立建模+自适应频段交互,显式捕捉频域特征
- 在两个数据集上优于现有模型4.9%,跨窗口长度均稳定领先
- 适合神经听觉系统、脑机接口等需要精准听觉解码的研究者
听觉注意力解码(AAD)旨在多说话人声学环境中从神经响应中推断出关注的说话人,是神经引导听力系统的关键问题。尽管近期研究取得进展,现有模型仍未能充分挖掘脑电图(EEG)的频域信息。多数方法通过手工特征提取或直接跨频段特征拼接引入多带信息,仅浅层利用频率信息,可能忽略频段特异性模式和跨频段交互。为此,本文提出FAConformer——一种面向频率的卷积-变压器框架,显式整合频段特异性编码与自适应跨频段交互。具体地,先将EEG信号分解为多个频段,每个频段交由独立的CNN-Transformer编码器进行频段特异性建模;随后,通过精心设计的频段感知注意力(FAA)模块,将各频段特征视为令牌进行自适应融合,建模跨频段依赖关系。此外,引入频段辅助监督(BAS),防止联合训练中贡献较弱分支被欠优化。由此,FAConformer实现更有效的频率感知建模。在两个公开的AAD数据集上,针对三种决策窗口长度的大量实验表明,该模型始终优于12个基线模型,超越当前最优模型4.9%。对频段重要性、消融实验及参数敏感性的分析验证了框架的有效性、鲁棒性与可解释性。代码已开源:https://github.com/wzwvv/FAConformer。
原文摘要 · Abstract (English)
Auditory attention decoding (AAD) aims to infer the attended speaker from neural responses in multi-speaker acoustic environments and is a key problem for neuro-steered hearing systems. Although recent studies have achieved encouraging progress, existing AAD models still do not fully exploit frequency domain electroencephalography (EEG) information. In particular, most approaches introduce multi-band information through handcrafted feature extraction or direct cross-band feature concatenation, which mainly exploit frequency information at a shallow level and may overlook band-specific patterns and cross-band interactions. To address these limitations, this paper proposes FAConformer, a frequency-aware CNN-Transformer framework for AAD that explicitly integrates band-specific encoding and adaptive cross-band interaction. Specifically, FAConformer first decomposes EEG signals into multiple frequency bands and assigns each band to an independent CNN-Transformer encoder for band-specific modeling. The resulting band-wise features are then adaptively fused by a carefully designed frequency-aware attention (FAA) module that models cross-band dependencies by treating band-wise features as tokens. Further, band-wise auxiliary supervision (BAS) is introduced to prevent weakly contributing branches from being under-optimized during joint training. In this way, FAConformer performs frequency-aware modeling that more effectively exploits frequency domain information. Extensive experiments on two public AAD datasets with three decision-window lengths demonstrated that FAConformer consistently outperformed 12 competitive baselines, surpassing the current state-of-the-art model by 4.9%. Further analyses of band importance, ablation, and parameter sensitivity verify the effectiveness, robustness, and interpretability of the proposed framework. Code is available at https://github.com/wzwvv/FAConformer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。