arXiv:2410.15615cs.CV2024-10中稿 · ICPR2024被引 9

提出新框架,高效精准定位3D场景中文本描述的物体。

Joint Top-Down and Bottom-Up Frameworks for 3D Visual Grounding

  • 用轻量模块生成粗略候选框,速度快
  • 通过图结构聚合上下文,精修候选框,准确率高
  • 适合需要实时与高精度的3D视觉定位任务

本文针对3D视觉定位任务——根据文本描述在点云场景中定位特定物体——提出一种联合自上而下与自下而上的框架。现有方法分为两类:自上而下依赖预训练3D检测器生成并筛选最优边界框,效率低;自下而上直接回归粗粒度特征的边界框,效果较差。为此,我们设计了一种双阶段框架:第一阶段采用基于自下而上的提案生成模块,通过轻量神经层高效回归并聚类多个粗略物体提案,避免复杂3D检测器;第二阶段引入自上而下的提案整合模块,利用图结构在生成的提案间有效聚合和传播与查询相关的对象上下文,实现进一步精修。两个模块联合训练,克服了自上而下框架中复杂提案的缺陷与自下而上框架中粗略提案的不足。在ScanRefer基准上的实验表明,该框架达到了当前最优性能。

原文摘要 · Abstract (English)

This paper tackles the challenging task of 3D visual grounding-locating a specific object in a 3D point cloud scene based on text descriptions. Existing methods fall into two categories: top-down and bottom-up methods. Top-down methods rely on a pre-trained 3D detector to generate and select the best bounding box, resulting in time-consuming processes. Bottom-up methods directly regress object bounding boxes with coarse-grained features, producing worse results. To combine their strengths while addressing their limitations, we propose a joint top-down and bottom-up framework, aiming to enhance the performance while improving the efficiency. Specifically, in the first stage, we propose a bottom-up based proposal generation module, which utilizes lightweight neural layers to efficiently regress and cluster several coarse object proposals instead of using a complex 3D detector. Then, in the second stage, we introduce a top-down based proposal consolidation module, which utilizes graph design to effectively aggregate and propagate the query-related object contexts among the generated proposals for further refinement. By jointly training these two modules, we can avoid the inherent drawbacks of the complex proposals in the top-down framework and the coarse proposals in the bottom-up framework. Experimental results on the ScanRefer benchmark show that our framework is able to achieve the state-of-the-art performance.

3D视觉定位点云理解目标检测多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。