arXiv:2602.01585cs.LG2026-02AAAI被引 6

用稀疏连接显式建模时序依赖,提升线性模型预测精度。

A Lightweight Sparse Interaction Network for Time Series Forecasting

  • 通过稀疏注意力机制显式捕捉时间步间重要关联。
  • 在多个公开数据集上超越先进线性与Transformer模型。
  • 结构轻量高效,适合资源受限的时序预测场景。

近期研究发现,线性模型在长时序预测任务中可超越多种Transformer模型。然而,传统线性模型依赖堆叠MLP隐式建模时间交互,难以充分捕捉复杂时序依赖,性能仍有提升空间。为此,我们提出轻量级稀疏交互网络LSINet。受自注意力稀疏性启发,设计多头稀疏交互机制(MSIM),通过伯努利分布诱导的稀疏性显式学习时间步间的关键连接。稀疏性由自适应正则化损失保障。同时观察到时序交互具有共享性,引入共享交互学习(SIL)以提升效率并加速收敛。LSINet为仅含MLP结构的线性模型,开销极低且具备显式时序交互能力。在多个公开数据集上的大量实验表明,其在准确率与效率方面均优于先进线性模型及Transformer模型。代码已开源:https://github.com/Meteor-Stars/LSINet。

原文摘要 · Abstract (English)

Recent work shows that linear models can outperform several transformer models in long-term time-series forecasting (TSF). However, instead of explicitly performing temporal interaction through self-attention, linear models implicitly perform it based on stacked MLP structures, which may be insufficient in capturing the complex temporal dependencies and their performance still has potential for improvement. To this end, we propose a Lightweight Sparse Interaction Network (LSINet) for TSF task. Inspired by the sparsity of self-attention, we propose a Multihead Sparse Interaction Mechanism (MSIM). Different from self-attention, MSIM learns the important connections between time steps through sparsity-induced Bernoulli distribution to capture temporal dependencies for TSF. The sparsity is ensured by the proposed self-adaptive regularization loss. Moreover, we observe the shareability of temporal interactions and propose to perform Shared Interaction Learning (SIL) for MSIM to further enhance efficiency and improve convergence. LSINet is a linear model comprising only MLP structures with low overhead and equipped with explicit temporal interaction mechanisms. Extensive experiments on public datasets show that LSINet achieves both higher accuracy and better efficiency than advanced linear models and transformer models in TSF tasks. The code is available at the link https://github.com/Meteor-Stars/LSINet.

时序预测稀疏建模轻量网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。