arXiv:2604.16988cs.LGcs.AI2026-04

Transformer模型能实时检测并适应数据分布突变,无需重新训练。

In-Context Learning Under Regime Change

论文配图:In-Context Learning Under Regime Change
图 1 · 摘自论文原文
  • 将突变检测建模为上下文学习问题,证明了Transformer可解决此任务。
  • 在不同已知程度的突变时间下,模型性能接近最优基准。
  • 无需重训即可提升真实场景下的疾病与金融预测表现。

非平稳序列在控制、预测和决策中自然出现,数据生成过程在未知时间发生转变,模型需识别突变、丢弃或降低旧证据权重,并实时适应新动态。基于Transformer的通用模型越来越多地依赖上下文学习进行时间序列预测、表格预测和连续控制。随着这些模型被部署于非平稳环境,理解其检测与适应突变的能力至关重要。本文将此问题形式化为上下文突变点检测问题,并正式证明存在可解决该问题的Transformer模型。其构建表明模型复杂度(层数与参数量)取决于对突变点位置的信息掌握程度,从完全未知到精确知晓。我们在合成线性回归和线性动态系统上验证结果,训练后的Transformer在各信息水平下性能匹配最优基线。此外,通过编码并引入突变点知识,无需重训即显著提升了预训练模型在传染病预测和美联储公开市场委员会(FOMC)公告前后金融波动预测中的表现,证实了其在真实世界突变场景中的实用性。

原文摘要 · Abstract (English)

Non-stationary sequences arise naturally in control, forecasting, and decision-making. The data-generating process shifts at unknown times, and models must detect the change, discard or downweight obsolete evidence, and adapt to new dynamics on the fly. Transformer-based foundation models increasingly rely on in-context learning for time series forecasting, tabular prediction, and continuous control. As these models are deployed in non-stationary environments, understanding their ability to detect and adapt to regime shifts is important. We formalize this as an in-context change-point detection problem and formally establish the existence of transformer models that solve this problem. Our construction demonstrates that model complexity, in layers and parameters, depends on the level of information available about the change-point location, from no knowledge to knowing exact timing. We validate our results with experiments on synthetic linear regression and linear dynamical systems, where trained transformers match the performance of optimal baselines across information levels. We also show that encoding and incorporating changepoint knowledge indeed improves the real-world performance of a pretrained foundation models on infectious disease forecasting and on financial volatility forecasting around Federal Open Market Committee (FOMC) announcements without retraining, demonstrating practical applicability to real-world regime changes.

时间序列Transformer突变检测自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。