提出并行处理不规则时间序列的液态门控注意力机制
Liquid Gated Attention

- 用观测时间间隔参数化输入驱动门控,实现连续时间建模
- 在长达17,984步的数据上保持线性计算复杂度
- 适合长时序、稀疏观测场景下的高效建模
现实世界的时间序列常呈现不规则采样和长时程特性,要求模型在任意时间间隔内捕捉连续动态,同时避免计算成本激增。离散时间方法将可变时间间隔压缩为静态位置步长;依赖求解器的连续时间模型虽保留时间结构,但需顺序积分,无法并行;而免求解器近似方法虽规避了此开销,却未将观测时间间隔与输入驱动状态调制相耦合。本文提出液态门控注意力(LGA),一种免求解器的并行时间算子。通过将输入驱动门控机制以观测时间间隔参数化,LGA引入连续时间归纳偏置,并将隐藏状态演化建模为快速权重关联记忆,实现时间维度上的并行计算。在非因果编码中利用矩阵结合律,在因果编码中采用前缀扫描,使两种模式下序列长度均呈线性时间复杂度。序列级归一化限制累积时间衰减,保障长时程优化稳定性。基于LGA,我们构建了LFormer,一个用于连续时间表征学习的模块化骨干网络。在六项任务和十六个数据集(最长达17,984步)上,LFormer展现出对长距离依赖的建模能力、细粒度状态追踪及从稀疏噪声观测中重建轨迹的能力,且在性能上媲美最先进离散与连续时间基线,同时具备线性扩展效率。
原文摘要 · Abstract (English)
Real-world time series often exhibit irregular sampling and extended temporal horizons, requiring models to capture continuous-time dynamics across arbitrary intervals without prohibitive scaling costs. Discrete-time methods collapse variable time intervals into static positional steps; solver-dependent continuous-time models preserve temporal structure but rely on sequential integration, precluding parallelization; and solver-free approximations avoid this cost yet none couples observed time intervals with input-driven state modulation. We propose Liquid Gated Attention (LGA), a solver-free parallel temporal operator. By parameterizing an input-driven gating mechanism with observed time intervals, LGA introduces a continuous-time inductive bias and formulates hidden state evolution as a fast-weight associative memory, enabling parallel computation across the temporal dimension. Using matrix associativity in non-causal encoding and a prefix scan in causal encoding, LGA attains linear temporal complexity in sequence length in both modes. A sequence-level normalization bounds cumulative temporal decay for stable long-horizon optimization. Building on LGA, we instantiate LFormer, a modular backbone for continuous-time representation learning. Across six tasks and sixteen datasets spanning up to 17,984 steps, LFormer demonstrates long-range dependency modeling, fine-grained state tracking, and trajectory reconstruction from sparse and noisy observations, while delivering competitive performance against state-of-the-art discrete-time and continuous-time baselines with linear scaling efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。