实现实时可重光照的神经辐射场，支持真实物理光照渲染。
Baking Relightable NeRF for Real-time Direct/Indirect Illumination Rendering
- 用CNN处理直接光照，轻量哈希网格递归模拟间接光照。
- 在未见光照条件下实现毫秒级渲染，画质损失极小。
- 适合虚拟现实、游戏等需要动态光影的实时应用。
重光照（Relighting）能够在训练时未见的光照条件下生成新视角，是实现沉浸式逼真体验的关键。然而，实时重光照面临巨大计算挑战：渲染方程需分解形状与材质并进行可见性测试以建模阴影；对于间接光照，还需在每个二次表面点上执行渲染方程计算。为此，本文提出一种新方法：采用CNN渲染器计算直接光照所需的主要表面点和渲染参数；同时设计一种轻量级哈希网格渲染器，通过递归执行实现二次光线追踪过程。两个渲染器均通过从预训练教师模型中蒸馏学习，可在未见光照条件下实现毫秒级物理真实感渲染，且画质损失可忽略不计。
原文摘要 · Abstract (English)
Relighting, which synthesizes a novel view under a given lighting condition (unseen in training time), is a must feature for immersive photo-realistic experience. However, real-time relighting is challenging due to high computation cost of the rendering equation which requires shape and material decomposition and visibility test to model shadow. Additionally, for indirect illumination, additional computation of rendering equation on each secondary surface point (where reflection occurs) is required rendering real-time relighting challenging. We propose a novel method that executes a CNN renderer to compute primary surface points and rendering parameters, required for direct illumination. We also present a lightweight hash grid-based renderer, for indirect illumination, which is recursively executed to perform the secondary ray tracing process. Both renderers are trained in a distillation from a pre-trained teacher model and provide real-time physically-based rendering under unseen lighting condition at a negligible loss of rendering quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。