让机器人感知真实尺度的几何信息,解决重建结果忽大忽小的问题。
Beyond Relative Geometry: Metric-Aware Geometry Perception for Robotics

- 通过相机参数与深度图联合建模,实现可度量的真实尺度重建。
- 绝对误差从2.01米降至0.07米,相对精度仍保持高水平。
- 适配多种传感器配置,直接提升机器人抓取等任务表现。
近期具身模型越来越多地利用几何表示来增强空间推理与机器人操作能力。然而,现有重建方法仅恢复相对几何结构,尺度任意,导致预测的物体尺寸与空间距离在不同场景、视角和输入配置下不一致,无法与真实世界尺度的机器人动作对齐。为此,我们提出度量感知几何感知(MAGP),一个端到端、即插即用的度量几何重建框架,可无缝集成至机器人策略中。核心在于度量尺度等变增强,使模型能根据相机参数与深度观测重建符合真实尺度的几何结构;灵活度量条件化则支持任意视图数量及相机与深度输入的组合,提升对异构传感配置的鲁棒性。两者结合确保几何重建在不同场景与感知条件下具有稳定的一致性。在ETH3D、MegaDepth和ScanNet++上的实验表明,MAGP在保持强相对几何精度的同时,将绝对误差降低一个数量级以上,由2.01米降至0.07米。集成至多个机器人策略后,其在LIBERO、RoboTwin和零样本LIBERO-Plus上均持续提升性能,最大增益达6.26%(在RoboTwin上)。结果验证了度量几何在机器人操作中的有效性与通用性。
原文摘要 · Abstract (English)
Recent embodied models increasingly leverage geometric representations to improve spatial reasoning and robotic manipulation. However, existing reconstruction methods only reconstruct relative geometry with arbitrary scales, causing predicted object dimensions and spatial distances to vary across scenes, viewpoints, and input configurations. This inconsistency prevents geometric perception from being directly aligned with robotic actions defined on the real-world scale. To address this limitation, we propose Metric-Aware Geometry Perception (MAGP), an end-to-end, plug-and-play framework for metric geometry reconstruction that can be seamlessly integrated into robotic policies. At its core, Metric Scale Equivariant Augmentation encourages the model to reconstruct metric geometry from camera parameters and depth observations, ensuring that the reconstructed geometry follows the metric scale specified by observations. Flexible Metric Conditioning further enables MAGP to support arbitrary view counts and combinations of camera and depth inputs, improving robustness to heterogeneous robotic sensing configurations. Together, these designs produce geometrically consistent reconstructions with stable object dimensions and spatial distances across scenes and sensing conditions. Experiments on ETH3D, MegaDepth, and ScanNet++ demonstrate that MAGP maintains strong relative geometry accuracy while reducing the absolute error by over an order of magnitude, from 2.01m to 0.07m. When integrated into multiple robotic policies, MAGP consistently improves performance on LIBERO, RoboTwin, and zero-shot LIBERO-Plus, with gains of up to 6.26% on RoboTwin. These results demonstrate the effectiveness and generalizability of metric geometry for robotic manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。