提出新训练目标，让世界模型更准确捕捉物理状态与动作影响。
PhyLatent: Learning Dynamics-Relevant Representations for JEPA World Models
- 通过物理状态对齐、反事实分支分离等三路径提升表征质量
- 在Cube任务上将失败率降至4.62%，预测控制成功率提至78.1%
- 适合研究物理建模、强化学习中的世界模型开发者
我们提出PhyLatent，一种针对联合嵌入预测架构（JEPA）世界模型的动力学相关训练目标。关键发现是：避免全局隐变量崩溃，并不保证表征能保留物理状态和动作后果。我们识别出JEPA世界模型的三种失效模式：物理不变性崩溃、物理可辨识性崩溃和反事实动力学崩溃。PhyLatent通过三条训练路径解决：物理状态对齐、未来表示对齐、静态视觉不变性、反事实分支分离与隐变量去噪。在OGBench-Cube上，三种失效率分别从15.60%、6.71%、8.41%降至7.53%、0.95%、4.62%，模型预测控制（MPC）成功率由70.0%提升至78.1%。相同架构与规划器下，在TwoRooms上成功率从81.0%提升至98.0%，在Reacher和PushT上仍保持竞争力。结果表明，仅避免全局非崩溃不足以学习可靠的JEPA世界模型状态空间。
原文摘要 · Abstract (English)
We propose PhyLatent, a dynamics-relevant training objective for JointEmbedding Predictive Architecture (JEPA) world models. Our key observation is that preventing global latent collapse does not ensure that a representation preserves physical states and action consequences. We identify three failure modes in JEPA world models: physical invariance collapse, physical identifiability collapse, and counterfactual dynamics collapse. PhyLatent addresses them through three training pathways: physical invariance, physical identifiability, and counterfactual dynamics, implemented with physical state grounding, future representation alignment, static visual invariance, counterfactual branch separation, and latent denoising. On OGBench-Cube, PhyLatent reduces the three failure rates from 15.60%, 6.71%, and 8.41% to 7.53%, 0.95%, and 4.62%, respectively, and improves model predictive control (MPC) success from 70.0% to 78.1%. With the same architecture and planner, it further improves success from 81.0% to 98.0% on TwoRooms and remains competitive on Reacher and PushT. These results show that global non-collapse alone is insufficient for learning a reliable JEPA worldmodel state space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。