用扩散模型指导对比学习，提升3D点云表征效果
PointDico: Contrastive 3D Representation Learning Guided by Diffusion Models
- 以扩散模型为指导，融合对比与生成学习优势
- 在ScanObjectNN上达94.32%准确率，ShapeNetPart上86.5%实例mIoU
- 适合研究3D表示学习、点云分析的开发者参考
自监督表征学习在自然语言处理和2D计算机视觉中表现优异，但面对无序且密度不均的3D数据时仍具挑战。现有方法中，对比模型易过拟合，3D掩码自编码器难以处理无序点云。为此，本文提出PointDico，通过知识蒸馏将扩散模型与对比学习无缝结合：扩散模型作为指导，驱动对比模型学习。设计分层金字塔条件生成器实现多尺度几何特征提取，并采用双通道结构有效融合局部与全局上下文信息。PointDico在3D表征学习上达到新基准，如在ScanObjectNN上取得94.32%准确率，在ShapeNetPart上实现86.5%的实例mIoU。
原文摘要 · Abstract (English)
Self-supervised representation learning has shown significant improvement in Natural Language Processing and 2D Computer Vision. However, existing methods face difficulties in representing 3D data because of its unordered and uneven density. Through an in-depth analysis of mainstream contrastive and generative approaches, we find that contrastive models tend to suffer from overfitting, while 3D Mask Autoencoders struggle to handle unordered point clouds. This motivates us to learn 3D representations by sharing the merits of diffusion and contrast models, which is non-trivial due to the pattern difference between the two paradigms. In this paper, we propose \textit{PointDico}, a novel model that seamlessly integrates these methods. \textit{PointDico} learns from both denoising generative modeling and cross-modal contrastive learning through knowledge distillation, where the diffusion model serves as a guide for the contrastive model. We introduce a hierarchical pyramid conditional generator for multi-scale geometric feature extraction and employ a dual-channel design to effectively integrate local and global contextual information. \textit{PointDico} achieves a new state-of-the-art in 3D representation learning, \textit{e.g.}, \textbf{94.32\%} accuracy on ScanObjectNN, \textbf{86.5\%} Inst. mIoU on ShapeNetPart.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。