首个可统一处理各类点云的3D预训练方法，提升几何结构学习效果
UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting
- 用高斯原语预测作为预训练任务，结合可微分渲染实现像素级监督
- 引入预训练图像模型的2D特征，增强纹理知识对几何结构的引导作用
- 适用于物体与场景级点云，适配多种3D模型架构，通用性强
点云数据尺度差异大，给统一的3D表征学习带来挑战。目前缺乏统一的3D模型，且无预训练方法能同时有效适用于物体级和场景级点云。本文提出UniPre3D，首个可无缝应用于任意尺度点云及任意架构3D模型的统一预训练方法。通过预测高斯原语作为预训练任务，并利用可微分高斯点绘（differentiable Gaussian splatting）生成图像，实现精确的像素级监督与端到端优化。为进一步调控预训练任务复杂度并引导模型关注几何结构，我们融合预训练图像模型的2D特征，引入成熟的纹理知识。在多种物体级与场景级任务上，使用不同点云模型作为骨干网络的大量实验验证了该方法的通用有效性。代码已公开于https://github.com/wangzy22/UniPre3D。
原文摘要 · Abstract (English)
The scale diversity of point cloud data presents significant challenges in developing unified representation learning techniques for 3D vision. Currently, there are few unified 3D models, and no existing pre-training method is equally effective for both object- and scene-level point clouds. In this paper, we introduce UniPre3D, the first unified pre-training method that can be seamlessly applied to point clouds of any scale and 3D models of any architecture. Our approach predicts Gaussian primitives as the pre-training task and employs differentiable Gaussian splatting to render images, enabling precise pixel-level supervision and end-to-end optimization. To further regulate the complexity of the pre-training task and direct the model's focus toward geometric structures, we integrate 2D features from pre-trained image models to incorporate well-established texture knowledge. We validate the universal effectiveness of our proposed method through extensive experiments across a variety of object- and scene-level tasks, using diverse point cloud models as backbones. Code is available at https://github.com/wangzy22/UniPre3D.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。