研究模型如何在噪声中推断隐藏的波动率状态,发现其内部存在两阶段计算机制。
Emergent Latent-State Computation under Stochastic Volatility

- 通过多变量随机波动率设置,观察模型如何从收益率中推断隐藏状态。
- Transformer 模型在长周期下形成可解释的线性投影+ℓ²归一化滤波器。
- 适合对模型可解释性、金融时间序列建模感兴趣的读者。
机制可解释性研究主要集中在语言模型和确定性简化任务上。对于序列模型在部分可观测、噪声环境下的潜在随机动态表示,了解甚少。本文在可控的多变量随机波动率设定中研究此问题,模型仅观测收益,而真实隐含波动率状态对研究者可见。该设定为部分可观测下的机制可解释性提供了良好基准:隐状态对模型隐藏,但可直接用于评估。在不同架构、损失函数和输出头下,均发现两阶段计算的证据:隐层表示包含下一隐含波动率状态的丰富信息,输出头将此表示映射为平方收益预测。此外,在 Transformer 中,隐状态可解码性在可识别的架构阶段出现,其位置取决于波动周期。在长周期情形下,该计算简化为一个显式隐状态滤波器,由学习到的线性投影加ℓ²归一化构成。输出头替换实验表明,噪声 MSE 训练下的性能下降部分源于读出失配,而非表征失败。结果表明,随机波动率模型为噪声潜态动态与部分可观测条件下的机制可解释性提供了有用基准。
原文摘要 · Abstract (English)
Mechanistic interpretability has largely focused on language models and deterministic toy tasks. Much less is known about how sequence models internally represent latent stochastic dynamics under noisy, partially observed observations. We study this question in a controlled multivariate stochastic volatility setting, where models observe only returns while the ground-truth latent volatility state is known to the researcher. This setting provides a useful benchmark for mechanistic interpretability under partial observability: the latent state is hidden from the model but directly available for evaluation. Across architectures, losses, and output heads, we find evidence for a two-stage computation. Hidden representations encode substantial information about the next latent volatility state, and the output head maps this representation to squared return forecasts. Furthermore, in Transformers, latent-state decodability emerges at identifiable architectural stages whose location depends on the volatility period. In long-cycle regimes, this computation simplifies into an explicit latent-state filter consisting of a learned linear projection followed by $\ell^2$ normalization. Output-head replacement further shows that part of the degradation under noisy MSE training arises from readout misalignment rather than representation failure. These results suggest that stochastic volatility models provide a useful benchmark for mechanistic interpretability under noisy latent dynamics and partial observability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。