用三维重建统一视觉与物理仿真，提升四足机器人真实部署效果。
QuadVerse: An Integrated Framework Aligning Visual-Physical Reality for Quadruped Simulation
- 基于RGB视频重建3DGS场景，支持逼真视角渲染与碰撞检测
- 通过轨迹优化校准摩擦参数，降低地形与执行器误差累积
- 实现无需实机微调的零样本视觉导航，适合机器人仿真部署
仿真在机器人学习中至关重要，但仿真到现实的差距仍是主要瓶颈。现有方法通常分别处理视觉或动态差异，忽视了这些误差在机器人状态演化过程中的累积与传播。本文提出QuadVerse，一个整合框架，利用重建场景作为校准基础，对齐视觉感知、物理交互与执行器动力学。从捕获的RGB视频中，我们重建几何约束的3D高斯溅射（3DGS）场景，支持批量逼真自视角渲染和可碰撞的语义网格提取。这些网格进一步通过轨迹驱动的后验搜索初始化并优化空间变化的摩擦先验，实现接触校准。为解决剩余执行器差异，QuadVerse在已校准地形上回放真实轨迹训练残差动力学补偿器，降低地形引起的接触误差与执行器非理想性之间的耦合。实验表明，QuadVerse在重建质量和运动追踪方面优于相关基线。在此基础上，我们展示了无需任务特定实机试跑的零样本视觉导航策略部署。
原文摘要 · Abstract (English)
Simulation is central to robot learning, yet the sim-to-real gap remains a major bottleneck. Existing approaches often tackle visual or dynamic gaps separately, overlooking how these individual mismatches accumulate and propagate throughout the robot's state evolution. In this paper, we introduce QuadVerse, an integrated framework that uses reconstructed scenes as a calibration substrate for aligning visual perception, physical interaction, and actuator dynamics. From captured RGB videos, we reconstruct geometry-constrained 3D Gaussian Splatting (3DGS) scenes that support batched photorealistic ego-view rendering and collision-ready semantic mesh extraction. The meshes further enable contact calibration by initializing spatially varying friction priors and refining them through trajectory-based posterior search. To address remaining actuator discrepancies, QuadVerse trains a residual dynamics compensator by replaying real-world trajectories on the contact-calibrated terrain, reducing the entanglement between terrain-induced contact errors and actuator non-idealities. Experiments show that QuadVerse improves reconstruction quality and locomotion tracking over relevant baselines. Leveraging this foundation, we demonstrate robust zero-shot visual-navigation policy deployment without task-specific real-world rollouts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。