通过聚类重建提升点云视觉模型的无监督预训练效果
Masked Clustering Prediction for Unsupervised Point Cloud Pre-training
- 用聚类预测代替传统掩码重建，学习更密集的语义特征
- 在多个3D任务上达到新最好结果，如分割和检测性能显著提升
- 适合做点云理解的无监督预训练，尤其适用于缺乏标注数据场景
视觉变换器（ViTs）在三维点云理解中广泛应用，以掩码自编码作为主流预训练范式。然而，标准ViTs从点云中学习密集且信息丰富的语义特征仍面临挑战。本文提出MaskClu，一种面向3D点云的新型无监督预训练方法，将掩码点建模与基于聚类的学习相结合。MaskClu旨在从被掩码的点云中重建聚类分配和聚类中心，从而促使模型捕捉更密集的语义信息。此外，引入全局对比学习机制，通过对同一点云的不同掩码视图进行对比，增强实例级特征学习。通过联合优化这些互补目标——密集语义重建与实例级对比学习，MaskClu使ViTs能够从3D点云中学习到更丰富、更具语义意义的表示。我们在多个3D任务（包括部分分割、语义分割、目标检测和分类）上验证了该方法的有效性，结果表明MaskClu达到了新的竞争性水平。代码与模型将发布于：https://github.com/Amazingren/maskclu。
原文摘要 · Abstract (English)
Vision transformers (ViTs) have recently been widely applied to 3D point cloud understanding, with masked autoencoding as the predominant pre-training paradigm. However, the challenge of learning dense and informative semantic features from point clouds via standard ViTs remains underexplored. We propose MaskClu, a novel unsupervised pre-training method for ViTs on 3D point clouds that integrates masked point modeling with clustering-based learning. MaskClu is designed to reconstruct both cluster assignments and cluster centers from masked point clouds, thus encouraging the model to capture dense semantic information. Additionally, we introduce a global contrastive learning mechanism that enhances instance-level feature learning by contrasting different masked views of the same point cloud. By jointly optimizing these complementary objectives, i.e., dense semantic reconstruction, and instance-level contrastive learning. MaskClu enables ViTs to learn richer and more semantically meaningful representations from 3D point clouds. We validate the effectiveness of our method via multiple 3D tasks, including part segmentation, semantic segmentation, object detection, and classification, where MaskClu sets new competitive results. The code and models will be released at:https://github.com/Amazingren/maskclu.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。