arXiv:2609.08788cs.LGcs.AI2026-09

为脑电等轴向信号设计可学习的注意力机制,提升模型性能。

Adaptive Anisotropic Attention for Axis-Structured Signals

论文配图:Adaptive Anisotropic Attention for Axis-Structured Signals
图 1 · 摘自论文原文
  • 将注意力分解为时间与空间两条路径,通过门控融合输出。
  • 在6个脑电任务中,平均平衡准确率显著优于传统注意力。
  • 适用于低信噪比的轴向信号,如脑电、音频频谱图。

密集自注意力在学习前对所有标记对一视同仁,这种交互同质性假设与结构化信号不匹配。对于脑电等低信噪比轴向信号,依赖关系沿电极和时间轴组织,此均质先验使每个标记暴露于大量无关交互。本文提出自适应各向异性注意力(AAA),将注意力分为两个路径:时间路径(同一电极跨时间)与空间路径(同一时间点其他电极)。通过一个小门控网络为每个标记预测两路径输出的凸组合(非负权重和为1)。在六个脑电下游任务中,所提出的模型AXON(轴分解算子网络)在线性探测与全微调下均优于密集基线。我们证明两条路径均必要,且加权融合优于硬选择;大部分收益来自门控在每层学习不同的时空平衡。受控音频频谱实验表明轴分解可迁移至脑电之外。结果表明,将注意力对齐信号自然轴向,能提供有效归纳偏置。

原文摘要 · Abstract (English)

Dense self-attention treats all token pairs as equally plausible before learning, an interaction-isotropic prior that can be mismatched to structured signals. For structured, low signal-to-noise ratio (SNR) signals such as EEG, dependencies are organized along the electrode and time axes, and this uniform prior exposes each token to many irrelevant interactions. We introduce Adaptive Anisotropic Attention (AAA), which splits attention into two paths: a temporal path, where each token attends to the tokens of its own electrode across time, and a spatial path, where it attends to the tokens of the other electrodes at the same time step. A small gate predicts, for every token, a convex combination of the two path outputs: two non-negative weights that sum to one. On six EEG downstream tasks, the resulting model, AXON (AXis-factorized Operator Network), improves mean balanced accuracy over a dense baseline under both linear probing and full fine-tuning. We show that both paths (temporal and spatial) are necessary and that the weighted sum beats a hard choice of one path; most of the benefit comes from the gate learning a different temporal/spatial balance at each layer of the network. Controlled audio spectrogram experiments show that axis factorization transfers beyond EEG. These results suggest that aligning attention with the natural axes of structured signals provides a useful inductive bias.

脑电分析注意力机制轴向建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。