提出解耦坐标系选择与定位预测的新方法,提升多视角3D定位准确率。
CoordRefer: Coordinate-Aware 3D Visual Grounding from Multiview Images

- 分离坐标系选择与3D框回归,避免数值歧义
- 在ScanRefer上提升11%的0.25精度和7%的0.5精度
- 适合需要高精度3D定位的视觉导航与机器人应用
基于多视角图像的3D视觉定位需先确定坐标系,再回归3D边界框进行定位。然而现有方法联合优化坐标系选择与框回归,导致坐标相对框存在歧义,降低定位性能。这是因为同一框在不同坐标系下可有多种数值表示,形成多个优化目标,产生无效妥协预测。为此,我们提出CoordRefer,一种坐标感知框架,将坐标系选择与坐标条件下的定位解耦。首先选定参考坐标系定义系统,再以该坐标系为条件进行3D框预测。通过坐标感知的监督微调建立坐标系选择与坐标条件框回归,并采用基于3D IoU的分组相对策略优化,对齐两阶段与下游定位质量。在ScanRefer数据集上使用Qwen3-VL-2B模型,CoordRefer相比无坐标感知基线,准确率提升11%([email protected])和7%([email protected]),其几何优化变体更优于使用显式3D输入的方法。
原文摘要 · Abstract (English)
Multiview image-based 3D visual grounding predicts a coordinate frame to define a coordinate system and then regresses a 3D bounding box for localization. However, existing methods jointly optimize coordinate frame selection and box regression, leading to coordinate-relative box ambiguity and degraded grounding performance. This ambiguity arises because the same box admits different numerical representations across coordinate frames, creating multiple optimization targets and yielding invalid compromise predictions. To tackle this challenge, we propose CoordRefer, a coordinate-aware framework that decouples coordinate frame selection from coordinate-conditioned grounding. CoordRefer first selects a reference frame to define the coordinate system and then conditions 3D box prediction on the coordinate system. We perform coordinate-aware supervised fine-tuning to establish coordinate frame selection and coordinate-conditioned box regression, followed by Group Relative Policy Optimization with 3D IoU-based rewards to align both stages with downstream grounding quality. On ScanRefer with Qwen3-VL-2B, CoordRefer achieves gains of 11% in [email protected] and 7% in [email protected] over the coordinate-agnostic baseline, while its geometrically refined variant surpasses methods using explicit 3D inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。