仅用一段视频就能让机器人学会推开门并穿过，真实世界成功率超96%。
Video2DoorTraversal: Push Door Traversal via Simulated Door Twins
- 用单段视频重建可仿真的门模型，实现真实到虚拟的无缝迁移。
- 在5个真实门上平均成功率达96.57%，未见门零样本成功率80.95%。
- 全程推理在机器人本地完成，13秒内完成开门穿越全过程，适合移动操作机器人应用。
开门穿越是一项长时程的移动操作任务，需精准把手交互与底盘-机械臂协同控制。我们提出Video2DoorTraversal，一种基于单段真实视频的端到端仿真-现实框架，适用于轮腿式移动操作机器人。给定一段真实门的RGB视频，DoorTwin重建出具有真实几何与外观、实例对齐且可模拟的门孪生体。仿真回路中的智能体将恢复的运动结构转化为参数化技能程序，并通过迭代优化失败轨迹生成物理可执行演示。这些演示用于训练ArticuACT——一种双深度策略网络，利用机器人视角摄像头输入和交互感知监督，预测协调的底盘、机械臂与夹爪指令。所有感知与策略推理均在机载设备上完成，系统在5个真实门上平均成功率达96.57%，对结构相似但未见过的门实现80.95%零样本成功率，平均耗时约13秒完成完整推进、开门与穿越流程。
原文摘要 · Abstract (English)
Door opening and traversal is a long-horizon loco-manipulation task that requires precise handle interaction and coordinated base-arm control. We present Video2DoorTraversal, a single-video real-to-sim-to-real framework for wheel-legged mobile manipulators. Given one RGB video of a real door, DoorTwin reconstructs an instance-aligned, articulated, and simulation-ready door twin with realistic geometry and appearance. A simulation-in-the-loop agent converts the recovered articulation into a parameterized skill program and iteratively refines failed rollouts to generate physically executable demonstrations. These demonstrations are used to train ArticuACT, a dual-depth policy that predicts coordinated base, arm, and gripper commands using robot-centric camera conditioning and interaction-aware supervision. With all perception and policy inference running onboard, the system achieves a 96.57% average success rate across five real doors and an 80.95% zero-shot success rate on structurally similar unseen doors, while completing the full approach, opening, and traversal sequence in approximately 13s on average. Project Page: https://video2doortraversal.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。