用神经隐式表示提升开放世界新类别发现的准确性
NeurNCD: Novel Class Discovery via Implicit Neural Representation
- 以隐式神经表示替代传统3D分割图，融合语义嵌入与熵信息
- 在NYUv2和Replica数据集上显著优于现有方法，无需密集标注
- 适合需要低资源、自动化的3D场景理解任务
在开放世界设置中发现新类别对真实应用至关重要。传统显式表示（如物体描述符或3D分割图）受限于离散性、孔洞和噪声，影响新类别发现的准确性。为此，我们提出NeurNCD，首个通用且数据高效的新型类别发现框架，采用精心设计的Embedding-NeRF模型，并以KL散度替代传统3D分割图，用于聚合视觉嵌入空间中的语义嵌入与熵信息。该框架还整合了特征查询、特征调制和聚类等组件，促进预训练语义分割网络与隐式神经表示间的高效特征增强与信息交换。结果表明，该方法在无需密集标注数据或人工干预生成稀疏标签监督的情况下，即可在开闭世界设置中实现更优的分割性能。大量实验显示，其在NYUv2和Replica数据集上显著超越当前最先进方法。
原文摘要 · Abstract (English)
Discovering novel classes in open-world settings is crucial for real-world applications. Traditional explicit representations, such as object descriptors or 3D segmentation maps, are constrained by their discrete, hole-prone, and noisy nature, which hinders accurate novel class discovery. To address these challenges, we introduce NeurNCD, the first versatile and data-efficient framework for novel class discovery that employs the meticulously designed Embedding-NeRF model combined with KL divergence as a substitute for traditional explicit 3D segmentation maps to aggregate semantic embedding and entropy in visual embedding space. NeurNCD also integrates several key components, including feature query, feature modulation and clustering, facilitating efficient feature augmentation and information exchange between the pre-trained semantic segmentation network and implicit neural representations. As a result, our framework achieves superior segmentation performance in both open and closed-world settings without relying on densely labelled datasets for supervised training or human interaction to generate sparse label supervision. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches on the NYUv2 and Replica datasets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。