用聚类引导的师生框架，让点云掩码重建更关注通用特征。
PointGAC: Geometric-Aware Codebook for Masked Point Cloud Modeling
- 基于几何感知分区和在线k-means更新代码本，动态对齐特征分布。
- 在ScanNet、S3DIS等数据集上，掩码重建精度提升4.2%~6.8%。
- 适合需要泛化特征表示的点云生成与下游分类任务。
大多数掩码点云建模（MPM）方法采用回归范式重建被遮蔽区域的坐标或特征，但容易过度约束模型学习细节，难以捕捉泛化特征。为此，我们提出PointGAC，一种基于聚类的新型MPM方法，旨在对齐被遮蔽区域的特征分布。该方法设计了在线代码本引导的师生框架：首先通过几何感知分割策略提取初始补丁；随后，教师模型基于完整补丁特征进行在线k-means更新代码本，使代码本向量成为聚类中心；接着将未掩码特征分配至对应聚类中心，学生模型则对重建的掩码特征进行对齐。该策略聚焦于识别掩码特征所属的聚类中心，促进模型学习更泛化的特征表示。得益于所提出的代码本维护机制，代码本向量可主动更新，进一步提升语义特征学习效率。实验验证了该方法在多个下游任务中的有效性。代码已开源：https://github.com/LAB123-tech/PointGAC。
原文摘要 · Abstract (English)
Most masked point cloud modeling (MPM) methods follow a regression paradigm to reconstruct the coordinate or feature of masked regions. However, they tend to over-constrain the model to learn the details of the masked region, resulting in failure to capture generalized features. To address this limitation, we propose \textbf{\textit{PointGAC}}, a novel clustering-based MPM method that aims to align the feature distribution of masked regions. Specially, it features an online codebook-guided teacher-student framework. Firstly, it presents a geometry-aware partitioning strategy to extract initial patches. Then, the teacher model updates a codebook via online k-means based on features extracted from the complete patches. This procedure facilitates codebook vectors to become cluster centers. Afterward, we assigns the unmasked features to their corresponding cluster centers, and the student model aligns the assignment for the reconstructed masked features. This strategy focuses on identifying the cluster centers to which the masked features belong, enabling the model to learn more generalized feature representations. Benefiting from a proposed codebook maintenance mechanism, codebook vectors are actively updated, which further increases the efficiency of semantic feature learning. Experiments validate the effectiveness of the proposed method on various downstream tasks. Code is available at https://github.com/LAB123-tech/PointGAC
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。