用少量图像生成可任意改变光照的高质量3D模型
RelitLRM: Generative Relightable Radiance for Large Reconstruction Models
- 基于变压器的前馈模型，分离几何与外观重建
- 仅需4-8张图像即可生成逼真光影效果，速度远超传统方法
- 适合快速3D内容创作和真实感渲染场景
我们提出RelitLRM，一种大型重建模型（LRM），可在仅4-8张稀疏视角图像且光照未知的情况下，生成高保真高斯点云表示的3D物体在新光照下的外观。与以往需要密集采集和缓慢优化的逆向渲染方法不同，RelitLRM采用前馈式变压器架构，结合几何重构器与基于扩散的可重光外观生成器，端到端训练于多视角合成渲染数据。该设计有效解耦几何与外观，解决材质与光照的模糊性问题，并捕捉阴影与高光的多模态分布。实验表明，该方法在稀疏视图下实现与主流密集视图优化基线相当的重光效果，同时显著提升速度。项目页面：https://relit-lrm.github.io/
原文摘要 · Abstract (English)
We propose RelitLRM, a Large Reconstruction Model (LRM) for generating high-quality Gaussian splatting representations of 3D objects under novel illuminations from sparse (4-8) posed images captured under unknown static lighting. Unlike prior inverse rendering methods requiring dense captures and slow optimization, often causing artifacts like incorrect highlights or shadow baking, RelitLRM adopts a feed-forward transformer-based model with a novel combination of a geometry reconstructor and a relightable appearance generator based on diffusion. The model is trained end-to-end on synthetic multi-view renderings of objects under varying known illuminations. This architecture design enables to effectively decompose geometry and appearance, resolve the ambiguity between material and lighting, and capture the multi-modal distribution of shadows and specularity in the relit appearance. We show our sparse-view feed-forward RelitLRM offers competitive relighting results to state-of-the-art dense-view optimization-based baselines while being significantly faster. Our project page is available at: https://relit-lrm.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。