用无标签真实图像训练,让模型稳定估计物体固有颜色。
SAIL: Self-supervised Albedo Estimation from Real Images with a Latent Diffusion Model
- 在潜在空间中构建分解模型,分离光照与颜色成分
- 仅需在线公开的多光照图像即可训练,无需标注数据
- 生成结果在不同光照下一致,适合真实场景编辑
内在图像分解旨在将图像分解为固有反射率和阴影分量,分离出物体基础颜色以支持虚拟调光与场景编辑等下游任务。尽管基于学习的方法取得进展,但真实世界图像的内在图像分解仍面临巨大挑战,主要因标注真值数据稀缺。现有方法多依赖合成数据进行监督训练,泛化能力受限。自监督方法则常产生含反光、光照不一致的反射率图。为此,我们提出SAIL,一种从单视角真实图像中估计类反射率表示的方法。利用潜在扩散模型在无条件场景调光中的先验知识,作为反射率估计的代理目标。我们首次在潜在空间中完整定义内在图像分解,并引入正则项约束光照相关与无关成分。SAIL仅使用在线可获取的未标注多光照数据,即可预测在不同光照条件下稳定的反射率图,具备跨场景泛化能力。
原文摘要 · Abstract (English)
Intrinsic image decomposition aims at separating an image into its underlying albedo and shading components, isolating the base color from lighting effects to enable downstream applications such as virtual relighting and scene editing. Despite the rise and success of learning-based approaches, intrinsic image decomposition from real-world images remains a significant challenging task due to the scarcity of labeled ground-truth data. Most existing solutions rely on synthetic data as supervised setups, limiting their ability to generalize to real-world scenes. Self-supervised methods, on the other hand, often produce albedo maps that contain reflections and lack consistency under different lighting conditions. To address this, we propose SAIL, an approach designed to estimate albedo-like representations from single-view real-world images. We repurpose the prior knowledge of a latent diffusion model for unconditioned scene relighting as a surrogate objective for albedo estimation. To extract the albedo, we introduce a novel intrinsic image decomposition fully formulated in the latent space. To guide the training of our latent diffusion model, we introduce regularization terms that constrain both the lighting-dependent and independent components of our latent image decomposition. SAIL predicts stable albedo under varying lighting conditions and generalizes to multiple scenes, using only unlabeled multi-illumination data available online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。