通过自监督表面一致性,让点云神经表示在大幅变形下仍保持完整表面。
P-CORE: Self-Supervised Surface Consistency for Point-Based Neural Editing

- 用随机变形前后表面一致来约束点云表示,无需真实变形数据。
- 在合成数据集上零样本编辑效果优于现有方法,显著减少伪影。
- 适用于需要自由变形的3D重建与编辑场景,如真实世界场景处理。
神经渲染技术已实现高保真多视角三维场景重建,但自由形态非刚性形状编辑仍是重大挑战。基于点的神经表示因无固定拓扑连接,适合多视角重建,但在大形变下易出现孔洞和表面不连续。为此,本文提出一种新颖的自监督方法,使点云表示在无需真实变形图像的情况下适应大形变。核心思想是生成随机变形,并确保变形前后预测表面一致:变形后点云的表面预测应等于原始点云表面预测的变形结果。该方法被集成到基于注意力的点表示中,其使用可学习的点间插值核而非每个点周围的高斯核。该插值核能自适应大形变,无需增删点。实验在合成几何编辑基准(Neural Editor、Objaverse)上显示,本方法在零样本编辑中表现更优,伪影显著减少。定性结果在DTU和Mip-NeRF 360数据集上也验证了其在真实场景中的有效性。
原文摘要 · Abstract (English)
Advances in neural rendering have enabled high-fidelity multi-view reconstruction of 3D scenes. However, free-form non-rigid shape editing remains a significant challenge. Point-based neural representations are highly desirable for multi-view reconstruction because they lack fixed connectivity, which does not constrain the learned surface topology to that of the initialization. Yet this same property causes point-based representations to struggle with holes and surface discontinuities under large deformations. To address this, we propose a novel self-supervised method to enable point-based representations to adapt to large deformations without requiring ground truth multi-view images of deformed geometry. The key idea is to generate random deformations and to ensure consistency in the predicted surface before and after deformation. In particular, the surface prediction from the deformed point cloud should be the same as the deformation applied to the surface prediction from the original point cloud. We incorporate our approach into attention-based point representations, which differ from splatting-based point representations in their use of a learned interpolation kernel between points as opposed to a Gaussian kernel around each point. This learned interpolation kernel can learn to adapt to large deformations, without requiring addition or removal of points. We show that our framework significantly enhances its robustness to large deformations. Experiments on synthetic geometry editing benchmarks (Neural Editor, Objaverse) demonstrate that our approach outperforms existing point-based methods in zero-shot editing and significantly reduces artifacts. Furthermore, qualitative results on the DTU and Mip-NeRF 360 datasets demonstrate our method's effectiveness on real-world scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。