用多视角蒸馏让模型从残缺点云想象完整3D形状
Imagine with the Teacher: Complete Shape in a Multi-View Distillation Way
- 通过多视角知识蒸馏，让学生网络模仿教师网络的补全思路
- 在PCN、ShapeNet等数据集上达到优于现有方法的补全精度
- 适合做3D形状补全、点云生成的研究者和开发者参考
点云补全旨在从因遮挡、传感器限制或噪声导致的不完整观测中恢复物体的完整3D形状。当不完整点云丢失关键语义信息时，神经网络需基于输入推断缺失部分。传统方法采用自编码器架构，以不完整点云为输入，由真实完整形状监督训练，使模型在隐空间中自动完成形状想象。但这一映射过程仍缺乏可解释性。受知识蒸馏启发，本文提出一种新的多视角蒸馏点云补全网络（VD-PCN），通过多视角知识迁移提升补全能力。该方法充分利用2D像素的有序性、2D处理的灵活性及2D网络的强大特征提取能力。在PCN、ShapeNet55/34和MVP数据集上的大量实验验证了所提方法的有效性，无论是定量还是定性指标均表现优异。为促进研究进展，代码将公开发布。
原文摘要 · Abstract (English)
Point cloud completion aims to recover the completed 3D shape of an object from its partial observation caused by occlusion, sensor's limitation, noise, etc. When some key semantic information is lost in the incomplete point cloud, the neural network needs to infer the missing part based on the input information. Intuitively we would apply an autoencoder architecture to solve this kind of problem, which take the incomplete point cloud as input and is supervised by the ground truth. This process that develops model's imagination from incomplete shape to complete shape is done automatically in the latent space. But the knowledge for mapping from incomplete to complete still remains dark and could be further explored. Motivated by the knowledge distillation's teacher-student learning strategy, we design a knowledge transfer way for completing 3d shape. In this work, we propose a novel View Distillation Point Completion Network (VD-PCN), which solve the completion problem by a multi-view distillation way. The design methodology fully leverages the orderliness of 2d pixels, flexibleness of 2d processing and powerfulness of 2d network. Extensive evaluations on PCN, ShapeNet55/34, and MVP datasets confirm the effectiveness of our design and knowledge transfer strategy, both quantitatively and qualitatively. Committed to facilitate ongoing research, we will make our code publicly available.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。