arXiv:2510.07729cs.CV2025-10被引 1

用表面八面体探针实现高效3D物体与场景合成,支持实时渲染和逼真阴影。

ComGS: Efficient 3D Object-Scene Composition via Surface Octahedral Probes

  • 引入表面八面体探针,通过插值高效查询光照与遮挡信息,避免昂贵的光线追踪。
  • 在28帧/秒下实现高质量实时渲染,物体放置处阴影逼真,编辑仅需36秒。
  • 聚焦物体周围环境光照重建,简化复杂光照估计,适合快速3D内容创作。

高斯点阵(GS)可实现沉浸式渲染,但真实感3D物体-场景合成仍具挑战性。现有方法中,烘焙的外观与阴影信息导致组合时出现不一致。为此,需实现可重光照的物体重建与场景光照估计。当前基于高斯的逆向渲染多依赖光线追踪,效率低下。本文提出表面八面体探针(SOPs),存储光照与遮挡信息,通过插值实现高效3D查询,避免光线追踪,重建速度提升至少2倍,并支持高斯场景中的实时阴影计算。对于光照估计,传统方法难以建模复杂光传输,学习方法则依赖单图且视角敏感。本文观察到3D合成主要关注物体位置的外观与邻近阴影,因此简化为重建物体所在位置的360°辐射场,并微调扩散模型完成光照补全。基于此,提出ComGS框架,可在约28 FPS下实现高质量实时渲染,生成视觉协调、阴影生动的结果,编辑耗时仅36秒。代码与数据集已公开于https://nju-3dv.github.io/projects/ComGS/。

原文摘要 · Abstract (English)

Gaussian Splatting (GS) enables immersive rendering, but realistic 3D object-scene composition remains challenging. Baked appearance and shadow information in GS radiance fields cause inconsistencies when combining objects and scenes. Addressing this requires relightable object reconstruction and scene lighting estimation. For relightable object reconstruction, existing Gaussian-based inverse rendering methods often rely on ray tracing, leading to low efficiency. We introduce Surface Octahedral Probes (SOPs), which store lighting and occlusion information and allow efficient 3D querying via interpolation, avoiding expensive ray tracing. SOPs provide at least a 2x speedup in reconstruction and enable real-time shadow computation in Gaussian scenes. For lighting estimation, existing Gaussian-based inverse rendering methods struggle to model intricate light transport and often fail in complex scenes, while learning-based methods predict lighting from a single image and are viewpoint-sensitive. We observe that 3D object-scene composition primarily concerns the object's appearance and nearby shadows. Thus, we simplify the challenging task of full scene lighting estimation by focusing on the environment lighting at the object's placement. Specifically, we capture a 360 degrees reconstructed radiance field of the scene at the location and fine-tune a diffusion model to complete the lighting. Building on these advances, we propose ComGS, a novel 3D object-scene composition framework. Our method achieves high-quality, real-time rendering at around 28 FPS, produces visually harmonious results with vivid shadows, and requires only 36 seconds for editing. Code and dataset are available at https://nju-3dv.github.io/projects/ComGS/.

3D合成高斯点阵实时渲染光照估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。