只预测物体变化部分,提升物理预测准确率与效率
Modeling What Changes: Sparse, Residual World Models for Object-Centric Manipulation

- 用变化门控+残差头,仅更新移动物体状态
- 8个物体时参数少11倍,精度高4.6倍,F1达0.87
- 零样本迁移、少量数据即可训练,适合机器人控制
整体世界模型在每一步都预测整个状态,浪费资源重复预测静态场景并引入误差。本文探讨是否显式建模变化(每个物体的变更门控 + 仅对被标记物体进行扰动的残差头)是一种更高效且可解释的物理预测与控制先验。在从3到8个物体的MuJoCo桌面推物基准上,该稀疏/残差模型在参数量仅为8.6至11.1倍的情况下,下一帧姿态预测精度比密集多层感知机高出2.5至4.6倍,变化检测F1保持在0.80至0.87之间,而密集基线退化为零;跨物体数量零微调仍保持99.4%的F1保留率;仅需四分之一数据即可达到全数据下约90%的精度。自回归滚动中累积误差更小,始终贴近无运动基准,而密集模型持续漂移。在采样规划器中,仅预测模型失败(尽管真实模拟器能解决任务,确认规划器有效),但当稀疏模型在规划器访问的状态上进行特征化与训练后,开始成功规划(0.23±0.06成功率,三组种子),而密集模型在所有种子上均保持为零。建模变化而非重预测全场景,是面向物体中心物理智能的简单有效先验;代码、数据生成器及所有检查点将在发表后公开。
原文摘要 · Abstract (English)
Monolithic world models predict the entire next state at every step, spending capacity re-predicting the static majority of a scene and injecting error into it. We ask whether explicitly modeling change (a per-object change gate plus a residual delta head that perturbs only the objects the gate flags) is a more effective and interpretable bias for physical prediction and control. On a MuJoCo tabletop pushing benchmark scaling from 3 to 8 objects, the sparse/residual model predicts next-state poses 2.5 to 4.6 times more accurately than a dense multilayer perceptron at 8.6 to 11.1 times fewer parameters, sustains change-detection F1 of 0.80 to 0.87 where the dense baseline is degenerate, transfers across object counts with zero retraining (99.4 percent F1 retention), and reaches about 90 percent of its full-data accuracy with a quarter of the data. In autoregressive rollout it compounds far less error, hugging the no-motion floor while the dense model drifts. Finally, inside a sampling-based planner, prediction-only models fail (though a true-simulator oracle solves the task with the identical planner, confirming the planner is sound), but once featurized and trained for the states a planner visits, the sparse model begins to plan (0.23 plus or minus 0.06 success over three seeds) while the dense monolith stays at zero at every seed. Modeling what changes, rather than re-predicting the whole world, is a simple, effective bias for object-centric physical AI; code, data generators, and all checkpoints will be released upon publication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。