用扩散模型统一不同光照下的3D物体合成，让虚拟物体无缝融入真实场景。
Lighting-Consistent Object Transfer Across Radiance Fields
- 通过扩散模型修复不同光照下合成的3D物体图像
- 在合成、生成和真实数据混合训练集上实现高质量光影一致性
- 适合影视特效、建筑设计等需要精准物体迁移的场景
3D高斯点云（3DGS）广泛用于真实场景的捕获与渲染。将一个场景中的物体合成到另一个场景，在影视特效、建筑与室内设计、营销等领域有广泛应用。然而，直接从源场景提取物体并粘贴到目标场景会因光照差异导致结果不真实。为此，我们提出一种扩散模型，可将光照不一致的合成图像进行调和。该模型在包含合成、生成和真实数据的异构图像对数据集上训练（输入：光照不一致的合成图，输出：光照一致的图像）。完整3D解决方案允许用户从源场景提取物体，并合成至目标场景。随后渲染出含合成物体的目标场景视角图像，扩散模型对每幅图像进行光照调和，最终通过后优化步骤整合为3DGS表示。实验表明，本方法视觉效果出色，显著优于先前方法。
原文摘要 · Abstract (English)
3D Gaussian Splatting (3DGS) is widely used to capture and render real scenes. Compositing objects from one capture into another has applications in many domains, such as VFX, architecture and interior design, or marketing. However, extracting an object from a source scene and naively pasting it into a target scene will fail to produce realistic results due to the different lighting conditions between the two scenes. To address this problem, we introduce a diffusion model that harmonizes naively composited images with inconsistent lighting. The model is trained with a heterogeneous dataset of image pairs (inconsistent composite input, consistent output), combining synthetic, generated, and real data. Our complete 3D solution allows a user to extract an object from the source scene and composite it into the target scene. From this, the (inconsistent) views of the target scene with the composite object are rendered. Our diffusion model harmonizes each one of these views, which are finally consolidated in a 3DGS representation with a post-optimization step. Our method provides visually compelling results, making object transfer between 3DGS easy to use and significantly improving quality compared to previous methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。