自动搜索更适合时间序列预测的Transformer结构，效果更优且训练高效。
Learning Novel Transformer Architecture for Time-series Forecasting
- 用可微架构搜索方法自动寻找最优注意力与激活设计
- 在多个时间序列数据集上超越现有模型精度
- 适合需要高精度时序预测的研究与工业应用
尽管基于Transformer的模型在时间序列预测（TSP）任务中表现优异，但现有Transformer架构仍存在局限，且缺乏对替代架构的系统探索。为此，我们提出AutoFormer-TS，一个针对TSP任务定制的新型框架，采用全面的架构搜索空间。该框架引入一种改进的可微神经架构搜索方法（AB-DARTS），提升了对最优操作的识别能力。AutoFormer-TS系统性地探索了替代注意力机制、激活函数和编码操作，突破传统Transformer设计。大量实验表明，AutoFormer-TS在多个TSP基准测试中持续优于现有最先进模型，在保持合理训练效率的同时实现了更高预测精度。
原文摘要 · Abstract (English)
Despite the success of Transformer-based models in the time-series prediction (TSP) tasks, the existing Transformer architecture still face limitations and the literature lacks comprehensive explorations into alternative architectures. To address these challenges, we propose AutoFormer-TS, a novel framework that leverages a comprehensive search space for Transformer architectures tailored to TSP tasks. Our framework introduces a differentiable neural architecture search (DNAS) method, AB-DARTS, which improves upon existing DNAS approaches by enhancing the identification of optimal operations within the architecture. AutoFormer-TS systematically explores alternative attention mechanisms, activation functions, and encoding operations, moving beyond the traditional Transformer design. Extensive experiments demonstrate that AutoFormer-TS consistently outperforms state-of-the-art baselines across various TSP benchmarks, achieving superior forecasting accuracy while maintaining reasonable training efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。