针对数据分布随时间变化的挑战,提出分治式专家混合模型提升长期预测精度。
NEST: Tackling Dataset-Level Distribution Shifts via Regime-Oriented Mixture-of-Experts

- 基于时序内容与熵空间聚类,将数据划分为不同运行模式
- 专家通过动态注意力机制捕捉各模式特有演化规律,性能超越现有方法
- 适合复杂系统长期预测任务,尤其适用于多变环境下的时序建模
复杂系统中的长期精准预测常受数据层面分布偏移影响,即不同行为模式与系统状态演变导致多元时间序列动态变化。现有方法多关注局部时间变化,却未显式建模数据由多个独立运行模式构成的全局结构挑战。本文提出 NEST 框架,采用两阶段密集型专家混合架构,首先在具原则性的矩-熵空间中通过无监督聚类划分数据为不同运行模式;引入面向模式的路由机制,依据时序内容生成初始专家权重,并通过几何调制优化至模式中心;关键在于,各专家不作为整体预测器,而是以专用核函数形式,通过演化独特变量注意力模式来捕获特定模式下的动态特征。在异构网络流量与物理现象等多样化基准上的广泛评估表明,NEST 持续取得领先性能。代码与数据集已公开于 https://github.com/Aaralshin/NEST。
原文摘要 · Abstract (English)
Accurate long-term forecasting in complex systems is frequently compromised by dataset-level distribution shifts, where diverse underlying behavioral modes and evolving system states drive the dynamic multivariate time-series. While existing methods predominantly focus on local temporal shifts, they fail to explicitly model the global structural challenge where datasets are composites of distinct operational regimes. In this paper, we propose NEST, a specialized framework designed to model and recompose these evolving structures through a two-phase dense MoE architecture. NEST first facilitates structural specialization by partitioning the dataset into distinct operational regimes through unsupervised clustering in a principled moment-entropy space. We introduce a regime-oriented router mechanism that generates initial expert weights based on temporal content, subsequently refined through geometric modulation to regime centroids. Crucially, rather than acting as monolithic predictors, individual experts function as specialized kernels that capture regime-specific dynamics by evolving unique variate-attention patterns. Extensive evaluations on diverse benchmarks, including heterogeneous network traffic and physical phenomena, demonstrate that NEST consistently achieves state-of-the-art performance. Our code and datasets are available at https://github.com/Aaralshin/NEST
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。