用参考图将点云转为3D高斯,一次前向计算即完成高质量重建。
GaussianPainter: Painting Point Cloud into 3D Gaussians with Normal Guidance
- 通过表面法向约束高斯旋转,缩小参数空间提升预测稳定性。
- 单次前向传播生成高质量3D高斯,避免耗时的测试时优化。
- 融合多尺度三平面表征注入图像外观,适合快速3D内容生成。
本文提出GaussianPainter,首个将点云与参考图像结合生成3D高斯的方法。针对3D高斯喷溅中因旋转、各向异性尺度及球谐系数构成的大参数空间导致的非唯一性问题,该方法引入表面法向估计来确定每个点的高斯旋转方向,从而在受限空间内更稳定地预测其余参数。同时,设计外观注入模块,利用多尺度三平面表示将参考图像外观融入高斯场。该方法仅需一次前向传播即可实现高效且保真的3D高斯生成,显著提升3D内容创作的速度与鲁棒性。
原文摘要 · Abstract (English)
In this paper, we present GaussianPainter, the first method to paint a point cloud into 3D Gaussians given a reference image. GaussianPainter introduces an innovative feed-forward approach to overcome the limitations of time-consuming test-time optimization in 3D Gaussian splatting. Our method addresses a critical challenge in the field: the non-uniqueness problem inherent in the large parameter space of 3D Gaussian splatting. This space, encompassing rotation, anisotropic scales, and spherical harmonic coefficients, introduces the challenge of rendering similar images from substantially different Gaussian fields. As a result, feed-forward networks face instability when attempting to directly predict high-quality Gaussian fields, struggling to converge on consistent parameters for a given output. To address this issue, we propose to estimate a surface normal for each point to determine its Gaussian rotation. This strategy enables the network to effectively predict the remaining Gaussian parameters in the constrained space. We further enhance our approach with an appearance injection module, incorporating reference image appearance into Gaussian fields via a multiscale triplane representation. Our method successfully balances efficiency and fidelity in 3D Gaussian generation, achieving high-quality, diverse, and robust 3D content creation from point clouds in a single forward pass.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。