arXiv:2501.19084cs.CV2025-01TPAMI被引 12

用文本高效分割3D场景,速度与精度双提升。

Laser: Efficient Language-Guided Segmentation in Neural Radiance Fields

  • 通过特征蒸馏直接提取密集CLIP特征,简化流程。
  • 训练速度更快,分割边界更精确,颜色相似区域一致性更强。
  • 适合需要快速、精准文本引导分割的3D应用开发者。

本文提出一种利用CLIP特征蒸馏实现语言引导3D分割的方法。与依赖多尺度CLIP特征、受处理速度和存储限制的方法不同,本方法通过直接高效地蒸馏密集CLIP特征,实现基于文本的3D场景精确分割。为此,我们引入适配器模块,并采用自交叉训练策略缓解密集特征蒸馏中的噪声问题。为提升分割边缘精度,提出低秩瞬时查询注意力机制。为保证不同视角下颜色相似区域的分割一致性,将分割任务转化为标签体素分类任务。此外,设计简化文本增强策略,缓解CLIP特征与文本对应模糊性。大量实验表明,该方法在训练速度和性能上均优于现有最先进水平。代码已开源:https://github.com/xingy038/Laser.git。

原文摘要 · Abstract (English)

In this work, we propose a method that leverages CLIP feature distillation, achieving efficient 3D segmentation through language guidance. Unlike previous methods that rely on multi-scale CLIP features and are limited by processing speed and storage requirements, our approach aims to streamline the workflow by directly and effectively distilling dense CLIP features, thereby achieving precise segmentation of 3D scenes using text. To achieve this, we introduce an adapter module and mitigate the noise issue in the dense CLIP feature distillation process through a self-cross-training strategy. Moreover, to enhance the accuracy of segmentation edges, this work presents a low-rank transient query attention mechanism. To ensure the consistency of segmentation for similar colors under different viewpoints, we convert the segmentation task into a classification task through label volume, which significantly improves the consistency of segmentation in color-similar areas. We also propose a simplified text augmentation strategy to alleviate the issue of ambiguity in the correspondence between CLIP features and text. Extensive experimental results show that our method surpasses current state-of-the-art technologies in both training speed and performance. Our code is available on: https://github.com/xingy038/Laser.git.

3D分割语言引导CLIP神经辐射场

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。