用预训练匹配先验优化3D高斯点云，提升稀疏视角下的几何重建质量
Flow Distillation Sampling: Regularizing 3D Gaussians with Pre-trained Matching Priors
- 通过预训练模型生成光流先验，指导高斯点云在未观测视角的采样
- 在深度渲染与新视角合成任务中，几何精度和视觉质量显著优于现有方法
- 适合关注3D重建几何保真度的科研与工业应用者
3D高斯溅射（3DGS）虽实现高质量渲染且训练推理速度快，但其优化过程缺乏显式几何约束，在观测视角稀疏或缺失区域易导致几何重建不佳。本文提出流蒸馏采样（FDS），将预训练的匹配先验引入3DGS优化流程。FDS通过策略性采样，聚焦于输入视图邻近的未观测视图，利用匹配模型计算的先验光流（Prior Flow）引导3DGS几何计算出的辐射光流（Radiance Flow）。在深度渲染、网格重建和新视角合成任务上的全面实验表明，FDS显著优于当前最优方法。可解释性实验进一步揭示了FDS对几何精度与渲染质量的影响机制，为理解其性能提供依据。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) has achieved excellent rendering quality with fast training and rendering speed. However, its optimization process lacks explicit geometric constraints, leading to suboptimal geometric reconstruction in regions with sparse or no observational input views. In this work, we try to mitigate the issue by incorporating a pre-trained matching prior to the 3DGS optimization process. We introduce Flow Distillation Sampling (FDS), a technique that leverages pre-trained geometric knowledge to bolster the accuracy of the Gaussian radiance field. Our method employs a strategic sampling technique to target unobserved views adjacent to the input views, utilizing the optical flow calculated from the matching model (Prior Flow) to guide the flow analytically calculated from the 3DGS geometry (Radiance Flow). Comprehensive experiments in depth rendering, mesh reconstruction, and novel view synthesis showcase the significant advantages of FDS over state-of-the-art methods. Additionally, our interpretive experiments and analysis aim to shed light on the effects of FDS on geometric accuracy and rendering quality, potentially providing readers with insights into its performance. Project page: https://nju-3dv.github.io/projects/fds
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。