arXiv:2605.08833cs.AI2026-05中稿 · ICML

用分数阶递归架构提升长序列建模中对短期突变的敏感度。

FRACTAL: SSM with Fractional Recurrent Architecture for Computational Temporal Analysis of Long Sequences

论文配图:FRACTAL: SSM with Fractional Recurrent Architecture for Computational Temporal Analysis of Long Sequences
图 1 · 摘自论文原文
  • 引入分数阶测度理论优化记忆更新机制
  • 在Long Range Arena上达87.11%平均分,ListOps任务61.85%
  • 适合需要精准捕捉短期变化的长序列分析场景

有效的序列建模需平衡无限历史保留与对真实世界现象中常见短时突变的高分辨率检测。然而,现有基于高阶多项式投影算子(HiPPO)的状态空间模型(SSMs)面临关键权衡:均匀测度会稀释近期信息以保持时间尺度不变性,而指数测度则牺牲全局上下文以捕捉局部动态。本文提出分数阶递归架构(FRACTAL),将分数阶测度理论融入递归记忆更新,以解决此限制。通过推导具有解析特征谱性质和可调奇异性指数的投影算子,该方法增强对近期信号扰动的敏感性,同时保持编码尺度不变记忆动态的谱结构。这一理论创新在简化的对角化状态空间框架中实现,通过调节输入投影初始化,实现多尺度时序特征的同步捕获。FRACTAL在Long Range Arena基准上取得87.11%的平均得分,其中ListOps任务达61.85%,优于S5模型。

原文摘要 · Abstract (English)

Effective sequence modeling fundamentally requires balancing the retention of unbounded history with the high-resolution detection of abrupt short-term variations common in real-world phenomena. However, existing state space models (SSMs) relying on high-order polynomial projection operators (HiPPO) face a critical trade-off where uniform measures dilute recent information to maintain timescale invariance, while exponential measures sacrifice global context to capture local dynamics. This paper proposes a Fractional Recurrent Architecture for Computational Temporal Analysis of Long sequences (FRACTAL), a novel architecture integrating fractional measure theory into recursive memory updates to address this limitation. By deriving projection operators with analytically characterized spectral properties and a tunable singularity index, the proposed method amplifies sensitivity to recent signal perturbations while preserving the spectral structure that encodes scale-invariant memory dynamics. This theoretical innovation is instantiated within a simplified diagonalized state space framework by modulating input projection initialization to enable simultaneous capture of multi-scale temporal features. FRACTAL achieves an average score of 87.11\% on the Long Range Arena benchmark, including 61.85\% on the ListOps task, outperforming the S5 model.

序列建模状态空间分数阶

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。