提出新方法让世界模型在测试时动态调整预测器，提升对环境变化的适应能力。
EPM-JEPA: Operator-Side Experience Modulation in JEPA-Family World Models
- 通过低秩权重调制方式，将经验信息注入预测器权重，实现动态更新
- 在移动MNIST任务中，相比基线提升1.90%，但未显著优于残差注入法
- 揭示了模型动态过程中的三重机制，为后续改进提供理论基础
JEPA家族世界模型在测试时使用固定预测器，其权重不随分布偏移而自适应。本文对比两种引入累积经验的机制：操作符侧调制（EPM-JEPA），即用压缩经验表示生成低秩权重增量并作用于预测器权重；以及操作数侧注入（EI-JEPA），即把经验表示作为残差添加到隐藏状态。在预注册实验（Moving MNIST，重力偏移）中，EPM-JEPA（D_shift^{n=50} = 0.7848 ± 0.0078，三组种子）与EI-JEPA（0.8238）差异为4.74%——按既定标准属于无效结果。作为次级非预注册观察，EPM-JEPA相比无记忆基线（0.8000）提升1.90%，且各种子一致；而EI-JEPA表现低于基线，表明优势仅存在于权重级调制。主要贡献在于机制分析：D_shift^{n=50}轨迹反映三个独立动力学过程——缓冲区循环、EMA目标漂移，以及内在的LoRA稳定瞬态+0.021，并非收敛至平衡态。这些发现推动开发基于物理建模的PEM-JEPA，以解决该动态峰值限制。
原文摘要 · Abstract (English)
JEPA-family world models use a static predictor whose weights do not adapt when test-time dynamics diverge from training. We compare two mechanisms for incorporating accumulated experience into a JEPA predictor under distribution shift: operand-side injection, where a compressed experience representation is added as a residual to the predictor's hidden state (EI-JEPA), and operator-side modulation, where the same representation generates low-rank weight deltas via LoRA applied to the predictor's weights (EPM-JEPA). On a pre-registered comparison (Moving MNIST, gravity shift), EPM-JEPA (D_shift^{n=50} = 0.7848 +/- 0.0078, three seeds) differs from EI-JEPA (0.8238) by delta = 4.74% - Outcome C: a null result - by our stated criterion, a valid outcome. As a secondary, non-pre-registered observation, EPM-JEPA improves 1.90% over a no-memory baseline (0.8000), consistently across seeds, while EI-JEPA underperforms the baseline, indicating the benefit is specific to weight-level modulation. Our primary contribution is a mechanism analysis: the D_shift^{n=50} trajectory reflects three independent dynamical processes - buffer cycling, EMA target drift, and an intrinsic LoRA settling transient of +0.021 - rather than convergence to equilibrium. These findings motivate PEM-JEPA, a physics-grounded successor addressing this dynamical-peak limitation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。