从一张图同时生成物体材质、纹理和光照，解决外观混淆问题。
Under One Sun: Multi-Object Generative Perception of Materials and Illumination
- 通过多物体共享光照的特性，分步解耦材质、纹理与光照。
- 在真实场景下实现高保真材质与光照重建，纹理细节完整保留。
- 适合需要精准材质建模的3D渲染与数字孪生应用。
我们提出多对象生成感知（MultiGP），一种从单张图像中随机采样所有辐射度成分——反射率、纹理和光照——的生成式逆渲染方法。其核心思路是：尽管物体的纹理和反射率可能不同，但同一场景中的物体均受同一光照影响。MultiGP利用这一共性，基于四项关键技术贡献，从已知形状的单张图像中生成一致的反射率、纹理与光照样本：级联端到端架构，融合图像空间与角度空间解耦；协调扩散收敛调度，确保光照估计唯一一致；轴向注意力机制促进不同反射率物体间的交互；纹理提取控制网络，在保持纹理高频细节的同时实现与光照的解耦。实验表明，MultiGP有效利用多个物体外观的互补空间与频率特征，成功恢复个体纹理、反射率及共同光照。
原文摘要 · Abstract (English)
We introduce Multi-Object Generative Perception (MultiGP), a generative inverse rendering method for stochastic sampling of all radiometric constituents -- reflectance, texture, and illumination -- underlying object appearance from a single image. Our key idea to solve this inherently ambiguous radiometric disentanglement is to leverage the fact that while their texture and reflectance may differ, objects in the same scene are all lit by the same illumination. MultiGP exploits this consensus to produce samples of reflectance, texture, and illumination from a single image of known shapes based on four key technical contributions: a cascaded end-to-end architecture that combines image-space and angular-space disentanglement; Coordinated Scheduling for diffusion convergence to a single consistent illumination estimate; Axial Attention applied to facilitate ``cross-talk'' between objects of different reflectance; and a Texture Extraction ControlNet to preserve high-frequency texture details while ensuring decoupling from estimated lighting. Experimental results demonstrate that MultiGP effectively leverages the complementary spatial and frequency characteristics of multiple object appearances to recover individual texture and reflectance as well as the common illumination.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。