用视觉语言模型评估机器人动作安全性，实现安全规划。
ChronoDreamer: Action-Conditioned World Model as an Online Simulator for Robotic Planning
- 基于时空变换器预测未来画面与接触分布。
- 在DreamerBench上实现高保真非接触运动与合理接触预测。
- 结合大模型判断碰撞风险，适合复杂操作任务的机器人系统。
我们提出ChronoDreamer，一种用于高接触场景机器人操作的动作条件世界模型。给定自我中心RGB图像、接触图、动作和关节状态的历史序列，ChronoDreamer通过类似MaskGIT的掩码预测训练的时空变换器，预测未来的视频帧、接触分布和关节角度。接触信息以深度加权高斯点云图像形式编码，将3D力转换为相机对齐格式，适配视觉骨干网络。推理时，利用视觉-语言模型评估预测轨迹的碰撞可能性，实现执行前的安全动作拒绝采样。模型在DreamerBench数据集上训练与评估，该数据集由Project Chrono生成，包含刚性与柔体物体场景下的同步RGB、接触点云、本体感知及物理标注。定性结果表明，模型在无接触运动中保持空间一致性，生成合理的接触预测；而基于大模型的评判器可有效区分碰撞与非碰撞轨迹。
原文摘要 · Abstract (English)
We present ChronoDreamer, an action-conditioned world model for contact-rich robotic manipulation. Given a history of egocentric RGB frames, contact maps, actions, and joint states, ChronoDreamer predicts future video frames, contact distributions, and joint angles via a spatial-temporal transformer trained with MaskGIT-style masked prediction. Contact is encoded as depth-weighted Gaussian splat images that render 3D forces into a camera-aligned format suitable for vision backbones. At inference, predicted rollouts are evaluated by a vision-language model that reasons about collision likelihood, enabling rejection sampling of unsafe actions before execution. We train and evaluate on DreamerBench, a simulation dataset generated with Project Chrono that provides synchronized RGB, contact splat, proprioception, and physics annotations across rigid and deformable object scenarios. Qualitative results demonstrate that the model preserves spatial coherence during non-contact motion and generates plausible contact predictions, while the LLM-based judge distinguishes collision from non-collision trajectories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。