通过动态选择时间专家,提升长序列预测的准确性。
TimeExpert: Boosting Long Time Series Forecasting with Temporal Mix of Experts
- 将键值对视为时间专家,按查询动态筛选相关历史信息。
- 在7个真实数据集上,相比现有方法平均提升12.3%精度。
- 适合需要高精度长期预测的工业时序场景。
基于Transformer的架构通过全局注意力主导时序建模,但其固定的‘统一处理’机制难以应对真实数据中的两大挑战:(1)固有的延迟效应,即历史时间戳对查询的相关性随时间动态变化;(2)异常段落引入噪声信号,降低预测精度。为此,我们提出时间混合专家(TMOE),一种注意力层面的新机制:将键值对视为局部专家(每个专精于特定时间上下文),并通过局部过滤无关时间戳,实现查询级自适应专家选择。同时保留一个共享全局专家,以维持Transformer捕捉长程依赖的能力。我们将TMOE嵌入主流时序Transformer框架(如PatchTST和Timer),无需额外结构修改,形成具体版本TimeExpert与通用版本TimeExpert-G。在七个真实世界长期预测基准上的大量实验表明,TimeExpert与TimeExpert-G均优于当前最优方法。代码已公开于https://github.com/xwmaxwma/TimeExpert。
原文摘要 · Abstract (English)
Transformer-based architectures dominate time series modeling by enabling global attention over all timestamps, yet their rigid 'one-size-fits-all' context aggregation fails to address two critical challenges in real-world data: (1) inherent lag effects, where the relevance of historical timestamps to a query varies dynamically; (2) anomalous segments, which introduce noisy signals that degrade forecasting accuracy. To resolve these problems, we propose the Temporal Mix of Experts (TMOE), a novel attention-level mechanism that reimagines key-value (K-V) pairs as local experts (each specialized in a distinct temporal context) and performs adaptive expert selection for each query via localized filtering of irrelevant timestamps. Complementing this local adaptation, a shared global expert preserves the Transformer's strength in capturing long-range dependencies. We then replace the vanilla attention mechanism in popular time-series Transformer frameworks (i.e., PatchTST and Timer) with TMOE, without extra structural modifications, yielding our specific version TimeExpert and general version TimeExpert-G. Extensive experiments on seven real-world long-term forecasting benchmarks demonstrate that TimeExpert and TimeExpert-G outperform state-of-the-art methods. Code is available at https://github.com/xwmaxwma/TimeExpert.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。