用物理光照控制单图重打光,速度快质量高。
PIXLRelight: Controllable Relighting via Intrinsic Conditioning

- 通过图像内在属性统一物理渲染与生成模型
- 支持任意PBR光照设置,每图处理<0.1秒
- 适合需要快速高质量重打光的视觉设计场景
我们提出PIXLRelight,一种前馈式单图物理可控重打光方法。现有方法或控制能力有限(如仅靠文本或环境贴图),或在逆向与正向渲染链中累积误差,或需昂贵的逐图优化。核心思想是通过可从真实照片或物理渲染中获得的共同内在条件,连接基于物理的渲染(PBR)与学习图像合成。训练时,成对多光照照片被分解为反照率、漫反射阴影和非漫反射残差,作为模型条件;推理时,从输入图像粗略3D重建的路径追踪渲染下,计算相同条件,再由基于Transformer的神经渲染器对源图施加目标光照,通过像素级仿射调制保留细节。PIXLRelight实现任意PBR风格光照控制,达到当前最优重打光质量,单图处理时间低于0.1秒。代码与模型见https://mlfarinha.github.io/pixl-relight/
原文摘要 · Abstract (English)
We present PIXLRelight, a feed-forward approach for physically controllable single-image relighting. Existing methods either provide limited lighting control (e.g. through text or environment maps), accumulate errors when chaining inverse and forward rendering, or require costly per-image optimization. Our key idea is to bridge physically based rendering (PBR) and learned image synthesis through a shared intrinsic conditioning that can be obtained from either real photographs or PBR renders. At training time, paired multi-illumination photographs are decomposed into albedo, diffuse shading, and non-diffuse residuals, which condition the model. At inference time, the same conditioning is computed from a path-traced render of a coarse 3D reconstruction of the input under user-specified PBR lights. A transformer-based neural renderer then applies the target illumination to the source photograph, preserving fine image detail through a per-pixel affine modulation. PIXLRelight enables arbitrary PBR-style lighting control, achieves state-of-the-art relighting quality, and runs in under a tenth of a second per image. Code and models are available at https://mlfarinha.github.io/pixl-relight/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。