通过因果分析发现时序大模型的潜在偏差和失效模式。
Causal Analysis for Time Series Foundation Models
- 构建因果分析框架,干预合成数据生成器观察模型响应变化。
- 发现两个模型对趋势、周期模式较安全,但高估持续性且在突变模式下易崩溃。
- 适合关注时序大模型可靠性的开发者与应用选型者参考。
从专用时序模型转向时序基础模型,使单一模型服务于众多高风险预测任务,带来集中化风险。本文提出一种因果分析框架,通过干预参数化合成时间序列生成器,在其他条件不变下测量模型输出变化,评估模型对时序模式的保留能力。我们对Chronos-2和TimesFM-2.5进行了测试,覆盖六种典型时间序列模式。结果表明:两种模型在趋势和周期模式下表现安全;但均存在高估持续性的偏差,且在制度切换模式下均出现突发性失败;此外TimesFM-2.5在能量释放模式下也发生失效。回顾原始论文发现,这些现象可能源于预训练数据分布。研究最后提出模型改进建议、应用场景选择策略,并讨论局限与未来方向。
原文摘要 · Abstract (English)
Transitioning from bespoke time series models towards time series foundation models changes the relationship of model and application from one-to-one to one-to-many. This shift introduces concentration risk as many, potentially high-risk, forecasting applications are exposed to the same biases and failure modes of a single time series foundation model. At the same time, this centralization allows for economies of scale in model development and validation. In this study we investigate how biases and failure modes of time series foundation models can be identified before deployment. We propose a causal analysis framework to investigate the ability of a time series foundation model to preserve time series patterns. To achieve this, we intervene on parameterized synthetic time series generators and measure the corresponding change in model output under ceteris paribus conditions. We apply our causal analysis framework to Chronos-2 and TimesFM-2.5 and test them across six distinct time series patterns. We find safe configurations for trend and harmonic oscillation patterns. The results also indicate a bias in both models towards overestimating persistence, sudden failures for both models against the regime switch pattern and failure for TimesFM-2.5 against the energy-release pattern. Our review of the original works for both models indicates that the findings might be explained by the data used for pretraining. We conclude our study with suggestions for further model development, recommendations for application-specific model selection, and a discussion of limitations and further research directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。