arXiv:2602.18473cs.LGcs.AI2026-02中稿 · ICLR被引 4

提出新型中心化模块CoTAR,提升医学时序信号建模效果

Decentralized Attention Fails Centralized Signals: Rethinking Transformers for Medical Time Series

  • 用全局核心令牌替代分散注意力,实现信号集中聚合与分发
  • 在APAVA数据集上提升11.6%准确率,内存仅用33%,推理时间降为20%
  • 特别适合需要捕捉多通道同步特征的脑电、心电分析任务

医疗时序数据(如脑电图、心电图)的精准分析对疾病诊断至关重要。这类数据具有单通道内时序依赖和多通道间跨通道依赖两大特征。尽管基于Transformer的模型能有效捕捉时序依赖,却难以建模通道依赖,根源在于其分散式注意力机制与医学信号固有的集中式特性不匹配。为此,本文提出CoTAR(核心令牌聚合-重分布)模块,采用中心化MLP结构替代传统注意力:通过一个全局核心令牌作为中介,统一协调各令牌交互,实现集中式信息聚合与分配。该设计不仅更契合医学信号的集中性,还将计算复杂度从二次方降至线性。在五个基准数据集上的实验表明,本方法在效果与效率上均优于现有最优模型,在APAVA数据集上准确率提升11.6%,仅需33%内存与20%推理时间。代码与训练脚本已公开于https://github.com/Levi-Ackman/TeCh。

原文摘要 · Abstract (English)

Accurate analysis of medical time series (MedTS) data, such as electroencephalography (EEG) and electrocardiography (ECG), plays a pivotal role in healthcare applications, including the diagnosis of brain and heart diseases. MedTS data typically exhibit two critical patterns: temporal dependencies within individual channels and channel dependencies across multiple channels. While recent advances in deep learning have leveraged Transformer-based models to effectively capture temporal dependencies, they often struggle with modeling channel dependencies. This limitation stems from a structural mismatch: MedTS signals are inherently centralized, whereas the Transformer's attention mechanism is decentralized, making it less effective at capturing global synchronization and unified waveform patterns. To address this mismatch, we propose CoTAR (Core Token Aggregation-Redistribution), a centralized MLP-based module designed to replace decentralized attention. Instead of allowing all tokens to interact directly, as in standard attention, CoTAR introduces a global core token that serves as a proxy to facilitate inter-token interactions, thereby enforcing a centralized aggregation and redistribution strategy. This design not only better aligns with the centralized nature of MedTS signals but also reduces computational complexity from quadratic to linear. Experiments on five benchmarks validate the superiority of our method in both effectiveness and efficiency, achieving up to a 11.6% improvement on the APAVA dataset, while using only 33% of the memory and 20% of the inference time compared to the previous state of the art. Code and all training scripts are available at https://github.com/Levi-Ackman/TeCh.

医学时序Transformer注意力机制脑电图

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。