用时间结构引导专家路由,让模型更准更省参数。
AME-TS: Anchored Mixture-of-Experts for Time Series Forecasting

- 根据时间序列特征动态分配专家,实现精准计算。
- 小模型性能超越同类,大模型仍保持领先,激活参数少40%以上。
- 适合需要高效预测和可解释性的工业时序场景。
时序预测模型正通过大型Transformer架构不断扩展,但现有方法通常对所有序列采用统一密集计算路径,忽略了时序结构的显著差异。混合专家(MoE)提供了一种自然替代方案,支持条件计算,但标准MoE路由难以稳定实现专家专业化。本文提出AME-TS,一种基于结构引导的稀疏时序基础模型,将专家路由与可解释的时间结构对齐。AME-TS首先使用轻量级分段预测器估计序列级描述符(包括可预测性、季节性、趋势和稀疏性),并据此构建专家上的软结构先验。该先验在训练中指导令牌级路由,促进结构对齐的专业化。在GIFT-Eval基准上,AME-TS在不同模型规模下均表现出优异的准确率-效率权衡:小模型显著优于现有时序基础模型,大模型仍保持竞争力,且通过稀疏路由激活的参数量减少超40%。进一步实验表明,在M5数据集微调时,AME-TS学习到更具可解释的路由几何结构,且专家专业化稳定性远超标准MoE。结果表明,结构感知路由是实现稀疏专家模型在时序预测中优势的有效可靠方式。
原文摘要 · Abstract (English)
Time series forecasting models are increasingly scaled through large Transformer backbones, yet most existing approaches process all series through a shared dense computation path despite substantial heterogeneity in temporal structure. Mixture-of-Experts (MoE) offers a natural alternative by enabling conditional computation, but standard MoE routing leaves expert specialization weakly identified and often unstable during downstream adaptation. We propose AME-TS, a structure-guided sparse time series foundation model that aligns expert routing with interpretable temporal structure. AME-TS first uses a lightweight regime predictor to estimate series-level descriptors, including forecastability, seasonality, trend, and sparsity, and maps them to a soft structural prior over experts. This series-level prior guides token-level routing during training, encouraging structure-aligned specialization. On the GIFT-Eval benchmark, AME-TS delivers a strong accuracy-efficiency tradeoff across model scales: it substantially outperforms existing time series foundation models at small model scales and remains competitive with the strongest models at larger scales, while activating substantially fewer parameters through sparse routing. We further show that AME-TS learns more interpretable routing geometry and substantially more stable expert specialization than standard MoE during fine-tuning on the M5 dataset. These results suggest that structure-aware routing is an effective and reliable way to realize the benefits of sparse expert models for time series forecasting.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。