arXiv:2605.24322cs.CV2026-05被引 3

通过物理概念向量,无需训练即可控制视频模型的物理推理方向。

Causal Physics Steering in Video World Models via Concept Activation Vectors

论文配图:Causal Physics Steering in Video World Models via Concept Activation Vectors
图 1 · 摘自论文原文
  • 用物理涌现区的线性探针权重作概念激活向量,干预隐藏状态。
  • 在IntPhys上可稳定改变模型对物理合理性的判断方向。
  • 揭示物理与运动方向分离编码,不同物理原则占不同方向。

视频世界模型学习物理动态表征,但推理时控制其物理预期仍是难题。近期可解释性研究发现,在VideoMAE的中间变压器层存在一个物理涌现区(PEZ),其中物理合理性被独立于其他视觉特征表示。然而,该结构能否直接用于控制模型物理推理仍不明确。本文提出无需训练的物理引导方法:利用PEZ层线性探针的权重作为概念激活向量(CAV),在推理时注入隐藏状态,从而改变模型的物理预期而不修改任何模型权重。在IntPhys基准测试中,该干预可可靠地使模型的合理性判断朝正负方向偏移。效果仅在物理涌现区内施加时出现,表明相关物理表征是局部化的。进一步发现,物理与运动方向分离编码,不同直觉物理原则占据该表征空间中的不同方向。这些结果表明,VideoMAE中的物理推理不仅可读,还可直接操控。

原文摘要 · Abstract (English)

Video world models learn representations of physical dynamics, but controlling their physical expectations at inference time remains an open problem. Recent interpretability work identified a Physics Emergence Zone (PEZ), a group of middle transformer layers in VideoMAE where physical plausibility is represented separately from other visual features. However, it remained unclear whether this structure could be used to directly control the model's physics reasoning. We present physics steering, a training-free method that uses the weight vector of a linear probe at a PEZ layer as a Concept Activation Vector (CAV) and injects it into hidden states during inference. This shifts the model's physical expectations without changing any model weights. On the IntPhys benchmark, this intervention reliably shifts the model's plausibility judgment in either direction, depending on the steering sign. The effect appears only when the intervention is applied within the Physics Emergence Zone, suggesting that the relevant physics representation is localized there. We further find that physics is encoded separately from motion direction, and that different intuitive physics principles occupy distinct directions within this representation space. Together, these results show that physical reasoning in VideoMAE is not only readable, but also directly steerable.

视频生成物理推理可解释性扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。