针对时序数据异构性,提出双层联邦学习方法提升基础模型性能。
Bi-level Heterogeneous Learning for Time Series Foundation Models: A Federated Learning Approach
- 分域间与域内双层次建模异构性,优化时序特征表示
- 在多个基准上实现点预测与概率预测的持续领先
- 适合跨领域时序建模,尤其适用于数据分布不均场景
时序数据的异构性远高于视觉或语言任务,因不同领域间时间动态差异显著。现有从零训练时序基础模型(TSFM)的方法多采用混合批次策略合并大规模数据集,易引发梯度冲突,降低表征质量。为此,我们提出一种细粒度学习方法,从异构序列中提炼不变知识,减少跨域干扰。通过双层次异构建模:域间与域内差异。设计联邦学习框架,通过局部正则化强化域不变性和语义一致性以缓解域内冲突,并利用域感知聚合增强跨域协作以应对域间差异。在多样化基准上的实验表明,该方法训练的TSFM在点预测和概率预测上持续优于集中式与联邦基线,在零样本场景下也表现良好,为异构环境下从零训练TSFM提供了灵活路径。
原文摘要 · Abstract (English)
Heterogeneity in time series data is more pronounced than in vision or language, as temporal dynamics vary substantially across domains and tasks. Existing efforts on training time series foundation models (TSFMs) from scratch are often trained with mixed-batch strategies that merge large-scale datasets, which can cause gradient conflicts and degrade representation quality. To address this, we propose a fine-grained learning method that distills invariant knowledge from heterogeneous series while reducing cross-domain interference. We characterize heterogeneity at two levels: inter-domain and intra-domain. To tackle this bi-level heterogeneity, we design a federated learning method that mitigates intra-domain conflicts by enforcing domain-invariant and semantically consistent representations through local regularization, and addresses inter-domain discrepancies by enhancing cross-domain collaboration via domain-aware aggregation. Experiments across diverse benchmarks show that TSFMs trained with our method consistently outperform both centralized and federated TSFM baselines in point and probabilistic forecasting, while also achieving competitive zero-shot performance at scale, offering a flexible pathway for training TSFMs from scratch in heterogeneous environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。