arXiv:2502.10392cs.CVcs.LG2025-02CVPR被引 25

用文本引导剪枝提升3D视觉定位效率,速度翻倍且精度领先。

TSP3D: Text-guided Sparse Voxel Pruning for Efficient 3D Visual Grounding

  • 通过文本引导的稀疏体素剪枝,实现3D场景与文本特征高效交互。
  • 在ScanRefer上达[email protected]提升1.13,在NR3D/SR3D上分别领先2.6/3.2。
  • 适合追求实时推理性能的3D视觉定位应用,尤其适用于复杂场景。

本文提出一种高效的多层级卷积架构用于3D视觉定位。传统方法因采用两阶段或基于点的结构,难以满足实时推理需求。受3D目标检测中多层级全稀疏卷积成功的启发,我们构建了遵循该技术路线的新框架。然而,3D视觉定位任务要求3D场景表示与文本特征深度交互,而基于稀疏卷积的架构因体素特征量大,交互效率低。为此,我们提出文本引导剪枝(TGP)和基于补全的添加(CBA),通过渐进式区域剪枝与目标补全,实现高效融合。具体而言,TGP迭代稀疏化3D场景表示,利用交叉注意力高效交互体素特征与文本特征。为缓解剪枝对精细几何信息的影响,CBA以极低计算开销自适应修复过度剪枝区域。相比以往单阶段方法,本方法推理速度领先,比最快方法提升100% FPS;同时在准确率上达到顶尖水平,优于两阶段方法,在ScanRefer上[email protected]提升1.13,在NR3D和SR3D上分别提升2.6和3.2。代码已开源。

原文摘要 · Abstract (English)

In this paper, we propose an efficient multi-level convolution architecture for 3D visual grounding. Conventional methods are difficult to meet the requirements of real-time inference due to the two-stage or point-based architecture. Inspired by the success of multi-level fully sparse convolutional architecture in 3D object detection, we aim to build a new 3D visual grounding framework following this technical route. However, as in 3D visual grounding task the 3D scene representation should be deeply interacted with text features, sparse convolution-based architecture is inefficient for this interaction due to the large amount of voxel features. To this end, we propose text-guided pruning (TGP) and completion-based addition (CBA) to deeply fuse 3D scene representation and text features in an efficient way by gradual region pruning and target completion. Specifically, TGP iteratively sparsifies the 3D scene representation and thus efficiently interacts the voxel features with text features by cross-attention. To mitigate the affect of pruning on delicate geometric information, CBA adaptively fixes the over-pruned region by voxel completion with negligible computational overhead. Compared with previous single-stage methods, our method achieves top inference speed and surpasses previous fastest method by 100\% FPS. Our method also achieves state-of-the-art accuracy even compared with two-stage methods, with $+1.13$ lead of [email protected] on ScanRefer, and $+2.6$ and $+3.2$ leads on NR3D and SR3D respectively. The code is available at \href{https://github.com/GWxuan/TSP3D}{https://github.com/GWxuan/TSP3D}.

3D视觉稀疏卷积文本引导实时定位

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。