arXiv:2507.11261cs.CV2025-07ICCV被引 11

通过多视角分解提升复杂3D视觉定位的准确性

ViewSRD: 3D Visual Grounding via Structured Multi-View Decomposition

  • 将复杂多锚点查询拆解为单一锚点描述,明确位置关系
  • 利用跨模态一致视角令牌融合多视角文本与场景特征
  • 在复杂空间描述下显著优于现有方法,适合高精度3D定位任务

3D视觉定位旨在根据文本描述识别并定位3D空间中的物体。然而,现有方法在处理复杂多锚点查询时难以区分目标与锚点,且受视角差异影响,空间描述常出现不一致。为此,我们提出ViewSRD框架,将3D视觉定位建模为结构化的多视角分解过程。首先,简单关系解耦(SRD)模块将复杂多锚点查询重构为一系列目标导向的单锚点陈述,生成具有视角感知能力的结构化描述,明确位置关系。这些分解后的表示作为基础,由多视角文本-场景交互(Multi-TSI)模块通过共享的跨模态一致视角令牌(CCVTs)在多视角间整合文本与场景特征,以保留空间相关性。最后,文本-场景推理模块将多视角预测融合为统一且鲁棒的3D视觉定位结果。在多个3D视觉定位数据集上的实验表明,ViewSRD显著优于当前最优方法,尤其在需要精确空间区分的复杂查询中表现突出。代码已公开于https://github.com/visualjason/ViewSRD。

原文摘要 · Abstract (English)

3D visual grounding aims to identify and localize objects in a 3D space based on textual descriptions. However, existing methods struggle with disentangling targets from anchors in complex multi-anchor queries and resolving inconsistencies in spatial descriptions caused by perspective variations. To tackle these challenges, we propose ViewSRD, a framework that formulates 3D visual grounding as a structured multi-view decomposition process. First, the Simple Relation Decoupling (SRD) module restructures complex multi-anchor queries into a set of targeted single-anchor statements, generating a structured set of perspective-aware descriptions that clarify positional relationships. These decomposed representations serve as the foundation for the Multi-view Textual-Scene Interaction (Multi-TSI) module, which integrates textual and scene features across multiple viewpoints using shared, Cross-modal Consistent View Tokens (CCVTs) to preserve spatial correlations. Finally, a Textual-Scene Reasoning module synthesizes multi-view predictions into a unified and robust 3D visual grounding. Experiments on 3D visual grounding datasets show that ViewSRD significantly outperforms state-of-the-art methods, particularly in complex queries requiring precise spatial differentiation. Code is available at https://github.com/visualjason/ViewSRD.

3D视觉定位多视角理解文本-场景对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。