通过空间剪枝提升3D视觉定位精度与效率
PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding
- 用语言引导筛选关键空间区域，缩小搜索范围
- 在复杂场景下实现9/10基准上领先性能
- 适合需要高效精准定位的三维视觉任务
3D视觉定位旨在根据自然语言描述定位3D场景中的目标物体。现有方法通常对整个场景进行推理，导致预测模糊且计算成本高，尤其在杂乱环境中更为明显。我们观察到，许多指代表达依赖局部空间上下文，往往对应有限的空间区域而非整个场景。基于此，提出PruneGround，一种即插即用的3DVG框架，包含三个核心组件：首先，引入语言引导空间剪枝（LGSP），利用冻结的视觉语言模型识别语言相关区域，减少空间计算并缩小定位候选范围；其次，提出多视角条件描述重表述（MCDR），将复杂表达分解为简化的目标-锚点关系，并通过多视角推理补全缺失的空间线索；最后，设计LLM-Grounder，将预训练检测模型转换为语言条件下的定位模型，通过对剪枝区域内点云与语言表征的对齐实现精准定位。在三个主流点云基准上的大量实验表明，该方法在所有ScanRefer设置及9/10 Nr3D/Sr3D设置中达到当前最优性能。代码与模型已公开：https://github.com/leduckhai/PruneGround
原文摘要 · Abstract (English)
3D Visual Grounding (3DVG) aims to localize target objects in 3D scenes given natural language descriptions. Existing approaches typically perform reasoning over the entire scene, leading to ambiguous predictions and high computational cost, especially in cluttered environments. We observe that many referential expressions rely on local spatial context and often correspond to restricted spatial regions rather than the full scene. Motivated by this insight, we propose PruneGround, an effective plug-and-play framework for 3DVG built upon three key components. First, we introduce Language-Guided Spatial Pruning (LGSP), which leverages a frozen Vision Language Model (VLM) to identify language-relevant regions, thereby reducing spatial computation and grounding candidates in the narrower search space. Second, we propose MultiView-Conditioned Description Reformulation (MCDR), which decomposes complex expressions into simplified target-anchor relations and augments missing spatial cues through multi-view reasoning. Finally, we propose LLM-Grounder, which repurposes a detection-pretrained spatial LLM into a language-conditioned grounding model by aligning point cloud and linguistic representations within the pruned region. Extensive experiments on the three most popular point cloud benchmarks demonstrate that our method achieves state-of-the-art results on all three ScanRefer settings and on 9 out of 10 Nr3D/Sr3D settings. Code and models are publicly available: https://github.com/leduckhai/PruneGround
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。