arXiv:2409.19702cs.CVcs.GR2024-09CVPR被引 19

让3D高斯点云能自由换光照,支持毛绒等软边界物体。

RNG: Relightable Neural Gaussians

  • 用神经网络建模光照与视角对颜色的影响,不依赖传统渲染公式。
  • 训练仅需1.3小时,渲染达60帧/秒,阴影质量优于同类方法。
  • 适合需要动态调光的虚拟内容创作,如游戏、影视资产建模。

3D高斯点云(3DGS)在新视角合成任务中表现优异,但通常假设光照固定。对于形状模糊的物体(如毛发、织物),实现可调光的3D资产仍具挑战性,因光照、几何与材质的分解关系不明确,尤其当表面平滑假设或基于表面的解析着色模型不适用时。本文提出可调光神经高斯(RNG),一种基于3DGS的新框架,支持硬表面与软边界物体的任意光照调整,且无需依赖特定着色模型。通过将每个点的辐射率同时依赖于视角与光照方向,并引入阴影提示及深度精修网络以提升阴影精度。此外,提出混合前向-延迟拟合策略,在几何与外观质量间取得平衡。相比基于神经辐射场的先前方法,本方法训练速度显著加快(仅需1.3小时),渲染速度达60帧/秒,且生成的阴影质量超越同期3DGS方法。

原文摘要 · Abstract (English)

3D Gaussian Splatting (3DGS) has shown impressive results for the novel view synthesis task, where lighting is assumed to be fixed. However, creating relightable 3D assets, especially for objects with ill-defined shapes (fur, fabric, etc.), remains a challenging task. The decomposition between light, geometry, and material is ambiguous, especially if either smooth surface assumptions or surfacebased analytical shading models do not apply. We propose Relightable Neural Gaussians (RNG), a novel 3DGS-based framework that enables the relighting of objects with both hard surfaces or soft boundaries, while avoiding assumptions on the shading model. We condition the radiance at each point on both view and light directions. We also introduce a shadow cue, as well as a depth refinement network to improve shadow accuracy. Finally, we propose a hybrid forward-deferred fitting strategy to balance geometry and appearance quality. Our method achieves significantly faster training (1.3 hours) and rendering (60 frames per second) compared to a prior method based on neural radiance fields and produces higher-quality shadows than a concurrent 3DGS-based method. Project page: https://www.whois-jiahui.fun/project_pages/RNG.

3D生成可调光高斯点云阴影优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。