通过补全与生成联合任务，提升点云自监督学习效果
PointCG: Self-supervised Point Cloud Learning via Joint Completion and Generation
- 设计补全与图像生成双任务协同预训练框架
- 在多个下游任务中优于现有基线方法
- 适合从事3D视觉自监督学习的研究者
自监督点云学习的核心在于设计合适的预训练任务，以构建使编码器有效感知3D物体的预训练框架。本文将掩码点建模（MPM）和3D到2D生成两种主流方法整合为预训练框架中的预任务。利用这两种方法的空间感知能力和精确监督信号，解决其各自存在的缺陷：模糊的监督信号和对几何信息不敏感的问题。所提出的框架名为PointCG，包含隐藏点补全（HPC）模块和任意视角图像生成（AIG）模块。首先，通过移除隐藏点，从任意视角获取可见点作为输入；随后，HPC模块使用编码器提取输入特征并用解码器完成完整形状重建，AIG模块则基于可见点特征生成渲染图像。大量实验表明，该方法在多种下游任务中显著优于基线模型。代码将在论文录用后公开。
原文摘要 · Abstract (English)
The core of self-supervised point cloud learning lies in setting up appropriate pretext tasks, to construct a pre-training framework that enables the encoder to perceive 3D objects effectively. In this paper, we integrate two prevalent methods, masked point modeling (MPM) and 3D-to-2D generation, as pretext tasks within a pre-training framework. We leverage the spatial awareness and precise supervision offered by these two methods to address their respective limitations: ambiguous supervision signals and insensitivity to geometric information. Specifically, the proposed framework, abbreviated as PointCG, consists of a Hidden Point Completion (HPC) module and an Arbitrary-view Image Generation (AIG) module. We first capture visible points from arbitrary views as inputs by removing hidden points. Then, HPC extracts representations of the inputs with an encoder and completes the entire shape with a decoder, while AIG is used to generate rendered images based on the visible points' representations. Extensive experiments demonstrate the superiority of the proposed method over the baselines in various downstream tasks. Our code will be made available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。