arXiv:2504.06719cs.CVcs.AI2025-04CVPR被引 5

提出3D场景自监督学习新方法,实现无需微调的高性能通用特征提取。

Masked Scene Modeling: Narrowing the Gap Between Supervised and Self-Supervised Learning in 3D Scene Understanding

  • 基于掩码场景建模,通过逐层重建被遮蔽区域的深层特征。
  • 在点云线性探测任务中性能媲美有监督模型,超越现有自监督方法。
  • 适用于3D场景理解中的通用特征提取,尤其适合缺乏标注数据的场景。

自监督学习在2D计算机视觉中已能提供与有监督模型相当的通用特征,但在3D场景理解领域,自监督方法通常仅用于初始化权重,难以直接用于通用特征提取。本文提出一种专为评估3D自监督特征质量而设计的鲁棒评估协议,采用多分辨率层级模型的点级特征采样,构建丰富的表示以反映模型语义能力,支持线性探测与最近邻方法评估。同时,我们首次提出一个原生在3D空间训练的自监督模型,其基于掩码场景建模目标,以自底向上的方式重建被遮蔽块的深层特征,特别适配层次化3D模型。实验表明,该方法在仅使用现成特征进行线性探测时表现媲美有监督模型,且显著优于现有自监督方法。代码与模型见GitHub:https://github.com/phermosilla/msm。

原文摘要 · Abstract (English)

Self-supervised learning has transformed 2D computer vision by enabling models trained on large, unannotated datasets to provide versatile off-the-shelf features that perform similarly to models trained with labels. However, in 3D scene understanding, self-supervised methods are typically only used as a weight initialization step for task-specific fine-tuning, limiting their utility for general-purpose feature extraction. This paper addresses this shortcoming by proposing a robust evaluation protocol specifically designed to assess the quality of self-supervised features for 3D scene understanding. Our protocol uses multi-resolution feature sampling of hierarchical models to create rich point-level representations that capture the semantic capabilities of the model and, hence, are suitable for evaluation with linear probing and nearest-neighbor methods. Furthermore, we introduce the first self-supervised model that performs similarly to supervised models when only off-the-shelf features are used in a linear probing setup. In particular, our model is trained natively in 3D with a novel self-supervised approach based on a Masked Scene Modeling objective, which reconstructs deep features of masked patches in a bottom-up manner and is specifically tailored to hierarchical 3D models. Our experiments not only demonstrate that our method achieves competitive performance to supervised models, but also surpasses existing self-supervised approaches by a large margin. The model and training code can be found at our Github repository (https://github.com/phermosilla/msm).

3D理解自监督学习掩码建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。