arXiv:2604.07997cs.CV2026-04中稿 · CVPR被引 2

仅用少量样本实现动态室内3D物体的增量检测,突破标注依赖瓶颈。

Few-Shot Incremental 3D Object Detection in Dynamic Indoor Environments

  • 利用视觉语言模型挖掘未知物体,提取2D语义与3D框特征。
  • 通过空间权重机制过滤噪声,提升点与框特征的可靠性。
  • 融合多模态原型与门控机制,实现少样本下新类别精准识别。

增量式3D物体感知是实现动态室内环境中具身智能的关键步骤。然而,现有方法需大量新类别标注才能达到良好性能。为此,本文提出FI3Det框架,仅需少量新样本即可实现高效3D感知,通过视觉语言模型(VLM)学习未见类别的知识。FI3Det在基础阶段引入VLM引导的未知物体学习模块,利用VLM挖掘未知物体并提取包括2D语义特征和类无关3D边界框在内的综合表示。为缓解表示中的噪声,设计了基于空间位置与框内特征一致性的加权机制,重新调整点级与框级特征贡献。此外,提出门控多模态原型印记模块,将对齐的2D语义与3D几何特征构建类别原型,计算分类得分,并通过多模态门控机制融合以实现新物体检测。作为首个少样本增量3D检测框架,我们在ScanNet V2与SUN RGB-D两个数据集上建立了批处理与序列评估设置,结果表明FI3Det在各类基线方法上均取得显著且一致的提升。代码已开源:https://github.com/zyrant/FI3Det。

原文摘要 · Abstract (English)

Incremental 3D object perception is a critical step toward embodied intelligence in dynamic indoor environments. However, existing incremental 3D detection methods rely on extensive annotations of novel classes for satisfactory performance. To address this limitation, we propose FI3Det, a Few-shot Incremental 3D Detection framework that enables efficient 3D perception with only a few novel samples by leveraging vision-language models (VLMs) to learn knowledge of unseen categories. FI3Det introduces a VLM-guided unknown object learning module in the base stage to enhance perception of unseen categories. Specifically, it employs VLMs to mine unknown objects and extract comprehensive representations, including 2D semantic features and class-agnostic 3D bounding boxes. To mitigate noise in these representations, a weighting mechanism is further designed to re-weight the contributions of point- and box-level features based on their spatial locations and feature consistency within each box. Moreover, FI3Det proposes a gated multimodal prototype imprinting module, where category prototypes are constructed from aligned 2D semantic and 3D geometric features to compute classification scores, which are then fused via a multimodal gating mechanism for novel object detection. As the first framework for few-shot incremental 3D object detection, we establish both batch and sequential evaluation settings on two datasets, ScanNet V2 and SUN RGB-D, where FI3Det achieves strong and consistent improvements over baseline methods. Code is available at https://github.com/zyrant/FI3Det.

3D检测少样本增量学习视觉语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。