arXiv:2605.10330stat.MLcs.LG2026-05

通过专家损失融合提升时间序列预测的MoE模型,训练更快更准。

Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integration

论文配图:Fast Training of Mixture-of-Experts for Time Series Forecasting via Expert Loss Integration
图 1 · 摘自论文原文
  • 将专家专属损失直接融入训练目标,引导专家更好分工。
  • 在多领域数据上优于Transformer、WaveNet等主流模型,且训练效率更高。
  • 支持增量更新,无需重训全模型,适合实时场景使用。

我们提出一种新型自适应混合专家(MoE)框架,用于时间序列预测,通过将专家特异性损失信息直接引入训练过程,增强专家专业化。整体目标函数包含基础预测损失和专家特异性损失,使专家级预测误差与全局损失共同指导训练。该框架结合部分在线学习策略,支持门控机制和专家参数的增量更新,显著降低计算成本,无需重复全模型重训。通过专家级损失感知与高效在线优化的结合,该方法在学习效率和预测性能上均取得提升。在经济、旅游和能源等不同频率的数据集上的实证结果表明,该方法在预测精度和计算效率上普遍优于统计方法及先进神经网络模型(如Transformer、WaveNet)。消融实验进一步验证了专家损失融合策略的有效性,凸显其对预测性能的贡献。

原文摘要 · Abstract (English)

We propose a novel adaptive Mixture-of-Experts (MoE) framework for time series forecasting that enhances expert specialization by incorporating expert-specific loss information directly into the training process. Notably, the overall objective comprises the base forecasting loss and expert-specific losses, allowing expert-level prediction errors to jointly shape training alongside the global forecasting loss. This framework is further combined with a partial online learning strategy, enabling incremental updates of both the gating mechanism and expert parameters. This approach significantly reduces computational cost by eliminating the need for repeated full model retraining. By integrating expert-level loss awareness with efficient online optimization, the proposed method achieves improved learning efficiency while maintaining strong predictive performance. Empirical results across economic, tourism, and energy datasets with varying frequencies demonstrate that the proposed approach generally outperforms both statistical methods and state-of-the-art neural network models, such as Transformers and WaveNet, in forecasting accuracy and computational efficiency. Furthermore, ablation studies confirm the effectiveness of the expert-specific loss integration strategy, highlighting its contribution to enhancing predictive performance.

时间序列MoE在线学习预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。