用Transformer直接实现多视角物体重光照，无需估计材质属性。
RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting
- 基于视频基础模型构建前馈生成Transformer，通过交叉注意力注入光照信息。
- 在9万件物体、3.9万种光照的LOD数据集上训练，实现逼真重光照效果。
- 支持单图、多图及新视角重光照，零样本泛化能力强，适合图形渲染研究者。
图像重光照传统上依赖复杂的逆向渲染流程，易陷入病态优化；或使用单图生成模型，忽略理解三维几何与材质交互所需的多视角线索。为此，我们提出一种前馈生成Transformer，可直接实现单图与多图重光照，完全跳过显式内在属性估计。该架构源自视频基础模型，包含一个潜在光照模块，通过交叉注意力动态将目标环境贴图注入空间特征。同时，采用置换不变的位置编码，对无序多视角输入进行对称处理，避免序列偏差。为训练此数据驱动模型，我们构建了大规模的Laval Objaverse Dataset（LOD），包含9万件物体和3.9万种独特光照。大量实验表明，该方法在视觉质量、照片级重光照效果及跨单图、多图、新视角任务的零样本泛化能力方面均达到当前最优水平。
原文摘要 · Abstract (English)
Image relighting is traditionally tackled via complex inverse rendering pipelines, which suffer from ill-posed optimization, or single-image generative models that ignore crucial multi-view cues necessary for understanding 3D geometry and material interactions. To address these limitations, we introduce a feed-forward generative Transformer for direct single- and multi-view image relighting that entirely bypasses explicit intrinsic property estimation. Adapted from a video foundation model, our architecture features a latent illumination module that dynamically injects target environment maps into spatial features via cross-attention. Furthermore, we employ permutation-invariant positional encodings to symmetrically process unordered multi-view inputs without sequential bias. To train this robust data-driven model, we construct the massive Laval Objaverse Dataset (LOD), comprising 90K objects and 39K unique illuminations. Extensive experiments demonstrate state-of-the-art visual quality, photorealistic relighting quality, and strong zero-shot generalization across single-view, multi-view, and novel-view relighting tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。