TwinFormer用双层结构高效预测长序列时间数据，精度超越主流模型。
TwinFormer: A Dual-Level Transformer for Long-Sequence Time-Series Forecasting
- 分两阶段处理：先局部建模每段内部动态，再全局捕捉段间关系。
- 在8个领域34个任务中27次进入前二，17次最优，最大预测跨度720步。
- 轻量GRU聚合全局信息，适合需要高精度长时预测的场景。
TwinFormer是一种用于长序列时间序列预测的分层Transformer。它将输入划分为不重叠的时间片段，并分两阶段处理：(1) 局部Informer使用top-$k$稀疏注意力建模片段内动态，随后进行均值池化；(2) 全局Informer采用相同的top-$k$注意力捕捉片段间的长程依赖。一个轻量GRU对全局上下文化的片段标记进行聚合，实现直接多步预测。该架构实现线性$O(kLd)$的时间与内存复杂度。在来自六个不同领域的八个真实世界基准数据集（涵盖天气、股价、温度、电力消耗、电力、疾病）上，预测范围为96-720步，TwinFormer在34项任务中获得27次前二名。其中17次在MAE和RMSE上排名第一，10次排名第二。其表现持续优于PatchTST、iTransformer、FEDformer、Informer和原始Transformer。消融实验验证了top-$k$稀疏注意力优于ProbSparse，且GRU聚合有效。代码已开源：https://github.com/Mahimakumavat1205/TwinFormer。
原文摘要 · Abstract (English)
TwinFormer is a hierarchical Transformer for long-sequence time-series forecasting. It divides the input into non-overlapping temporal patches and processes them in two stages: (1) a Local Informer with top-$k$ Sparse Attention models intra-patch dynamics, followed by mean pooling; (2) a Global Informer captures long-range inter-patch dependencies using the same top-$k$ attention. A lightweight GRU aggregates the globally contextualized patch tokens for direct multi-horizon prediction. The resulting architecture achieves linear $O(kLd)$ time and memory complexity. On eight real-world benchmarking datasets from six different domains, including weather, stock price, temperature, power consumption, electricity, and disease, and forecasting horizons $96-720$, TwinFormer secures $27$ positions in the top two out of $34$. Out of the $27$, it achieves the best performance on MAE and RMSE at $17$ places and $10$ at the second-best place on MAE and RMSE. This consistently outperforms PatchTST, iTransformer, FEDformer, Informer, and vanilla Transformers. Ablations confirm the superiority of top-$k$ Sparse Attention over ProbSparse and the effectiveness of GRU-based aggregation. Code is available at this repository: https://github.com/Mahimakumavat1205/TwinFormer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。