用频域专家混合模型提升时间序列预测精度,兼顾周期性和先验模式。
MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models
- 在注意力后引入时域与频域专家,通过稀疏路由构建多维表示。
- 在6个公开数据集上比Time-MoE降低6.95% MSE和6.02% MAE。
- 适用于需捕捉复杂周期规律的商业场景,如真实销售数据预测。
时间序列预测作为重要数据模态任务,在诸多应用中具有关键作用。随着大语言模型(LLM)的显著进展,将LLM作为时间序列建模的基础架构受到广泛关注。尽管现有模型取得一定成效,但很少能在预训练-微调范式下同时建模时间与频率特性,导致复杂时间序列预测表现不佳,而这类任务需要同时捕捉周期性与信号先验模式。为此,本文提出MoFE-Time,一种融合时域与频域特征的新型时间序列预测模型,采用预训练-微调框架,实现跨不同周期分布数据集的先验模式知识有效迁移。该方法在注意力模块后引入时域与频域专家,并利用MoE路由机制构建输入信号的多维稀疏表示。在六个公开基准测试中,MoFE-Time达到新最优性能,相比代表性方法Time-MoE,MSE与MAE分别降低6.95%和6.02%。此外,我们还构建了源自真实商业场景的私有数据集NEV-sales,模型在此数据集上表现优异,验证了其在实际商业应用中的有效性。
原文摘要 · Abstract (English)
As a prominent data modality task, time series forecasting plays a pivotal role in diverse applications. With the remarkable advancements in Large Language Models (LLMs), the adoption of LLMs as the foundational architecture for time series modeling has gained significant attention. Although existing models achieve some success, they rarely both model time and frequency characteristics in a pretraining-finetuning paradigm leading to suboptimal performance in predictions of complex time series, which requires both modeling periodicity and prior pattern knowledge of signals. We propose MoFE-Time, an innovative time series forecasting model that integrates time and frequency domain features within a Mixture of Experts (MoE) network. Moreover, we use the pretraining-finetuning paradigm as our training framework to effectively transfer prior pattern knowledge across pretraining and finetuning datasets with different periodicity distributions. Our method introduces both frequency and time cells as experts after attention modules and leverages the MoE routing mechanism to construct multidimensional sparse representations of input signals. In experiments on six public benchmarks, MoFE-Time has achieved new state-of-the-art performance, reducing MSE and MAE by 6.95% and 6.02% compared to the representative methods Time-MoE. Beyond the existing evaluation benchmarks, we have developed a proprietary dataset, NEV-sales, derived from real-world business scenarios. Our method achieves outstanding results on this dataset, underscoring the effectiveness of the MoFE-Time model in practical commercial applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。