arXiv:2509.23695cs.LGcs.AI2025-09被引 2

用上下文学习预测时间序列模型迁移效果,省去盲目试错。

Estimating Time Series Foundation Model Transferability via In-Context Learning

  • 将模型选择转为上下文学习问题,利用源数据表现预测目标性能。
  • 在10个数据集上平均相关性达0.6,比零样本性能提升30%。
  • 适用于缺乏公开数据的领域,帮助快速选最优预训练模型。

时间序列基础模型(TSFMs)通过大规模预训练实现强大的零样本预测能力,但在公开数据有限的领域,微调仍是提升性能的关键。随着TSFM数量增加,高效筛选适合下游微调的最佳模型变得愈发困难。本文提出TimeTic框架,将模型选择重构为上下文学习问题:基于已知源数据集上的表现,预测某TSFM在目标数据集微调后的性能。TimeTic灵活组织模型-数据关系作为上下文信息,可适应多种测试场景。利用数据集元特征、模型特性与微调性能构成的自然表格结构,采用表格基础模型作为上下文学习器。我们还提出一种基于模型层间熵演化的新颖表征方法,捕捉嵌入空间差异,使TimeTic能跨任意模型集泛化。构建了包含10个数据集、10个基础模型和3个预测任务的综合性迁移能力评估基准。在该基准上,TimeTic的估计与实际微调性能高度一致,平均秩相关约为0.6,相较使用零样本性能作为迁移度量指标提升30%。

原文摘要 · Abstract (English)

Time series foundation models (TSFMs) offer strong zero-shot forecasting via large-scale pre-training, yet fine-tuning remains critical for boosting performance in domains with limited public data. With the growing number of TSFMs, efficiently identifying the best model for downstream fine-tuning becomes increasingly challenging. In this work, we introduce TimeTic, a transferability estimation framework that recasts model selection as an in-context-learning problem: given observations on known (source) datasets, it predicts how a TSFM will perform after fine-tuning on a downstream (target) dataset. TimeTic flexibly organizes the observed model-data relationships as contextual information, allowing it to adapt seamlessly to various test-time scenarios. Leveraging the natural tabular structure formed by dataset meta-features, model characteristics, and fine-tuned performance, we employ tabular foundation models to serve as in-context learners. We further introduce a novel model characterization based on entropy evolution across model layers, capturing embedding-space distinctions and enabling TimeTic to generalize across arbitrary model sets. We establish a comprehensive benchmark for transferability estimation including 10 datasets, 10 foundation models, and 3 forecasting tasks. On this benchmark, TimeTic's estimation demonstrates strong alignment with actual fine-tuned performance for previously unseen datasets, achieving a mean rank correlation of approximately 0.6 and a 30% improvement compared to using zero-shot performance as the transferability score.

时间序列模型选择迁移评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。