通过操控隐藏状态模拟金融暴跌等罕见事件,验证时序大模型具备语义理解能力。
time2time: Causal Intervention in Hidden States to Simulate Rare Events in Time Series Foundation Models
- 用激活移植法将崩溃期统计特征注入平静期隐藏状态,实现因果干预
- 模型隐层向量模长与系统性冲击幅度直接相关,体现事件严重程度的分级表征
- 适用于风险评估、压力测试,为可解释的‘假如’分析提供新范式
尽管基于Transformer的时序基础模型在预测常规模式上表现优异,但其是否内化了市场状态等语义概念,或仅是拟合曲线仍存疑问。本文提出激活移植这一因果干预方法,在前向传播中将某一事件(如历史崩盘)的统计矩施加到另一事件(如平稳期)的隐藏状态上。该操作可确定性地引导预测:注入崩盘语义导致下行预测,注入平稳语义则抑制崩盘并恢复稳定。超越二元控制,我们发现模型编码了事件严重程度的连续表征,隐层向量模长与系统性冲击大小直接相关。在两种架构不同的时序基础模型Toto(仅解码器)和Chronos(编码-解码器)上验证,结果表明可调节、语义基础的表示是大型时序Transformer的稳健特性。研究揭示了支配模型预测的潜在概念空间,推动可解释性从事后归因转向直接因果干预,支持战略压力测试中的语义‘假设分析’。
原文摘要 · Abstract (English)
While transformer-based foundation models excel at forecasting routine patterns, two questions remain: do they internalize semantic concepts such as market regimes, or merely fit curves? And can their internal representations be leveraged to simulate rare, high-stakes events such as market crashes? To investigate this, we introduce activation transplantation, a causal intervention that manipulates hidden states by imposing the statistical moments of one event (e.g., a historical crash) onto another (e.g., a calm period) during the forward pass. This procedure deterministically steers forecasts: injecting crash semantics induces downturn predictions, while injecting calm semantics suppresses crashes and restores stability. Beyond binary control, we find that models encode a graded notion of event severity, with the latent vector norm directly correlating with the magnitude of systemic shocks. Validated across two architecturally distinct TSFMs, Toto (decoder only) and Chronos (encoder-decoder), our results demonstrate that steerable, semantically grounded representations are a robust property of large time series transformers. Our findings provide evidence for a latent concept space that governs model predictions, shifting interpretability from post-hoc attribution to direct causal intervention, and enabling semantic "what-if" analysis for strategic stress-testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。