arXiv:2608.06827cs.ROcs.CV2026-08被引 1

用双代理机制补足稀疏采集下的仿真视觉,提升机器人真实到仿真迁移效果。

R2S-EGO: Dual-Proxy Refinement for Sparse-Capture Real-to-Sim

论文配图:R2S-EGO: Dual-Proxy Refinement for Sparse-Capture Real-to-Sim
图 1 · 摘自论文原文
  • 构建仿真代理与捕捉结构代理协同优化场景表示
  • 六视图下达到19.062 dB PSNR,显著优于基线14.226 dB
  • 适合需要高效真实数据采集的机器人仿真训练场景

真实到仿真(R2S)依赖于沿机器人自身轨迹渲染观测的场景表示,但密集多视角采集限制了每个环境的真实图像采集效率,而稀疏的人类采集又可能导致行为相关视角支持不足。相机控制的合成可填补缺失视角,但在R2S中使用需满足行为可接受的查询和捕捉锚定的结构约束。我们提出R2S-EGO,其耦合一个模拟器生成的机器人代理(代表行为相关可执行查询域)与一个捕捉锚定的几何代理(提供场景特定结构条件)。在此域内,固定预算的选择针对当前几何支持可用的支持缺陷进行优化。生成的观测被作为伪观测融入以精炼视觉资产,而真实采集仍作为锚点。融合的几何代理还提供场景碰撞表面,并在每轮中更新。这些更新共同在保持机器人动力学和控制栈不变的前提下精炼现有仿真场景。在三个Replica场景中48个冻结的Unitree G1自车视角上,六视图的R2S-EGO达到19.062 dB PSNR,优于最强报告基线的14.226 dB;在五组配对策略训练种子下,实现82.5% ± 6.8%的真实G1坐下成功率,远超GaussGym的10.0% ± 10.5%。

原文摘要 · Abstract (English)

Real-to-sim (R2S) depends on scene representations that render observations along robot ego trajectories, yet dense multi-view capture limits per-environment real-image capture-count efficiency, and sparse human capture can leave behavior-scoped robot views under-supported. Camera-controlled synthesis can fill missing views, but its use in R2S requires behavior-admissible queries and capture-anchored structural conditioning. We present R2S-EGO, which couples a simulator-derived robot proxy that represents the behavior-scoped executable query domain with a capture-anchored geometry proxy that supplies scene-specific structural conditions. Within this domain, fixed- budget selection targets current support deficits for which geometry support is available. The generated observations are assimilated as pseudo-observations to refine the visual asset, while real captures remain anchors. The fused geometry proxy also supplies the scene collision surface, which is refreshed between rounds. Together, these updates refine the existing simulation scene while its robot dynamics and control stack stay fixed. Across 48 frozen Unitree G1 ego views in three Replica scenes, six-view R2S-EGO reaches 19.062 dB PSNR, compared with 14.226 dB for the strongest reported R2S baseline. Across five paired policy-training seeds, R2S-EGO achieves 82.5% +/- 6.8% real-G1 sitting success, compared with 10.0% +/- 10.5% for GaussGym.

真实到仿真机器人仿真视觉补全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。