让机器人自选最佳视角，提升视觉模仿学习的鲁棒性。
Observer-Actor: Active Vision Imitation Learning with Sparse-View Gaussian Splatting
- 观察者自主移动到最优视角，构建3D高斯点云并虚拟探索最佳相机位姿。
- 在遮挡和无遮挡场景下，轨迹迁移性能提升超200%，行为克隆提升超75%。
- 适合双臂机器人视觉模仿学习，尤其对视角遮挡敏感的任务有显著优势。
我们提出观察者-执行者（ObAct）框架，用于主动视觉模仿学习，其中观察者会移动至最优视觉观测位置。研究基于配备腕部相机的双臂机器人系统。测试时，ObAct动态分配观察者与执行者角色：观察者机械臂从三张图像构建3D高斯点云（3DGS），虚拟探索以找到最优相机位姿，随后移动至该位置；执行者机械臂则利用观察者的观测执行策略。该方法增强了物体与夹爪在策略观测中的清晰度与可见性。因此，我们能够在更接近无遮挡训练分布的观测下训练双臂通用策略，从而提升策略鲁棒性。我们在两种现有模仿学习方法——轨迹迁移与行为克隆上验证该框架，实验显示，相比固定相机设置，轨迹迁移在无遮挡和有遮挡条件下分别提升145%和233%，行为克隆分别提升75%和143%。视频见 https://obact.github.io。
原文摘要 · Abstract (English)
We propose Observer Actor (ObAct), a novel framework for active vision imitation learning in which the observer moves to optimal visual observations for the actor. We study ObAct on a dual-arm robotic system equipped with wrist-mounted cameras. At test time, ObAct dynamically assigns observer and actor roles: the observer arm constructs a 3D Gaussian Splatting (3DGS) representation from three images, virtually explores this to find an optimal camera pose, then moves to this pose; the actor arm then executes a policy using the observer's observations. This formulation enhances the clarity and visibility of both the object and the gripper in the policy's observations. As a result, we enable the training of ambidextrous policies on observations that remain closer to the occlusion-free training distribution, leading to more robust policies. We study this formulation with two existing imitation learning methods -- trajectory transfer and behavior cloning -- and experiments show that ObAct significantly outperforms static-camera setups: trajectory transfer improves by 145% without occlusion and 233% with occlusion, while behavior cloning improves by 75% and 143%, respectively. Videos are available at https://obact.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。