arXiv:2602.18071cs.RO2026-02

让机器人用单眼摄像头完成复杂环境中的多物推移,无需全局定位。

EgoPush: Learning End-to-End Egocentric Multi-Object Rearrangement for Mobile Robots

  • 基于物体中心的隐空间编码相对位置关系,避免依赖易错的全局坐标。
  • 在仿真中成功率远超传统端到端强化学习方法,且可零样本迁移到真实机器人。
  • 适合研究移动机器人自主操作、视觉导航与少监督学习的开发者。

人类能利用第一视角感知,在杂乱环境中通过避障完成物体重排,无需全局坐标。受此启发,我们研究移动端机器人在单一第一视角摄像头下的长时程多物体非抓取重排任务。提出EgoPush框架,通过物体中心的隐空间表示物体间的相对空间关系,而非绝对位姿,从而实现无需显式全局状态估计的感知驱动重排。设计一个强化学习教师模型,从稀疏关键点联合学习隐状态与移动动作,并将其知识蒸馏为纯视觉的学生策略。为缩小教师(全知)与学生(部分观测)间的监督差距,限制教师仅使用视觉可及线索,诱导其产生可被学生视角恢复的主动感知行为。针对长时程信用分配问题,采用分阶段局部完成奖励,结合时间衰减机制分解任务。大量仿真实验表明,EgoPush显著优于端到端强化学习基线,在成功率上提升明显;消融实验验证了各设计的有效性。进一步在真实移动平台上实现零样本仿真到现实的迁移。代码与视频见https://ai4ce.github.io/EgoPush/。

原文摘要 · Abstract (English)

Humans can rearrange objects in cluttered environments using egocentric perception, navigating occlusions without global coordinates. Inspired by this capability, we study long-horizon multi-object non-prehensile rearrangement for mobile robots using a single egocentric camera. We introduce EgoPush, a policy learning framework that enables egocentric, perception-driven rearrangement without relying on explicit global state estimation that often fails in dynamic scenes. EgoPush designs an object-centric latent space to encode relative spatial relations among objects, rather than absolute poses. This design enables a privileged reinforcement-learning (RL) teacher to jointly learn latent states and mobile actions from sparse keypoints, which is then distilled into a purely visual student policy. To reduce the supervision gap between the omniscient teacher and the partially observed student, we restrict the teacher's observations to visually accessible cues. This induces active perception behaviors that are recoverable from the student's viewpoint. To address long-horizon credit assignment, we decompose rearrangement into stage-level subproblems using temporally decayed, stage-local completion rewards. Extensive simulation experiments demonstrate that EgoPush significantly outperforms end-to-end RL baselines in success rate, with ablation studies validating each design choice. We further demonstrate zero-shot sim-to-real transfer on a mobile platform in the real world. Code and videos are available at https://ai4ce.github.io/EgoPush/.

机器人操作视觉导航强化学习少样本学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。