让插入的3D物体光影真实自然,自动匹配场景光照。
LightHarmony3D: Harmonizing Illumination and Shadows for Object Insertion in 3D Gaussian Splatting
- 用生成模型一次预测全景光照贴图,避免反复优化。
- 在多个真实数据集上实现最高逼真度和视角一致性。
- 首个专为3DGS中物体插入设计的评测基准,适合虚拟制作人群。
3D高斯点阵(3DGS)能实现场景几何与外观的高保真重建。基于此能力,将外部网格物体插入重建的3DGS场景中,可支持增强现实/虚拟现实、虚拟布景及数字内容创作中的交互编辑与内容扩展。然而,实现插入物体在光照与阴影上的物理一致性仍具挑战,需准确估计场景光照并保证多视角渲染一致。为此,我们提出LightHarmony3D,一种面向3DGS场景中光照一致物体插入的新框架。核心是提出的生成模块,通过单次前向传播即可在插入位置预测完整的360° HDR环境贴图。利用生成先验而非迭代优化,该方法高效捕捉主导场景光照,实现物理可信的着色与阴影,并保持多视角一致性。此外,我们首次构建了针对3DGS中物体插入的专用评测基准,提供标准化评估体系以衡量光照一致性和照片级真实感。在多个真实世界重建数据集上的大量实验表明,LightHarmony3D在逼真度和多视角一致性方面达到当前最佳水平。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) enables high-fidelity reconstruction of scene geometry and appearance. Building on this capability, inserting external mesh objects into reconstructed 3DGS scenes enables interactive editing and content augmentation for immersive applications such as AR/VR, virtual staging, and digital content creation. However, achieving physically consistent lighting and shadows for mesh insertion remains challenging, as it requires accurate scene illumination estimation and multi-view consistent rendering. To address this challenge, we present LightHarmony3D, a novel framework for illumination-consistent mesh insertion in 3DGS scenes. Central to our approach is our proposed generative module that predicts a full 360° HDR environment map at the insertion location via a single forward pass. By leveraging generative priors instead of iterative optimization, our method efficiently captures dominant scene illumination and enables physically grounded shading and shadows for inserted meshes while maintaining multi-view coherence. Furthermore, we introduce the first dedicated benchmark for mesh insertion in 3DGS, providing a standardized evaluation framework for assessing lighting consistency and photorealism. Extensive experiments across multiple real-world reconstruction datasets demonstrate that LightHarmony3D achieves state-of-the-art realism and multi-view consistency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。