arXiv:2509.00777cs.GRcs.CV2025-09

让合成数据训练的图像分解模型更好适应真实世界。

IntrinsicReal: Adapting IntrinsicAnything from Synthetic to Real Objects

  • 用双伪标签策略筛选高质量输出,逐步优化模型在真实图像上的表现。
  • 在真实数据集上实现领先性能,显著优于现有方法。
  • 适合需要高精度图像分解的应用场景,如摄影修复、材质分析。

从真实环境拍摄的单张RGB图像中估计反照率(即内在图像分解)面临巨大挑战,因为缺乏配对图像及其真实反照率标签。尽管近期方法(如IntrinsicAnything)利用强大的扩散先验取得了突破,但仍主要在大规模合成数据集(如Objaverse)上训练,并直接应用于真实图像,忽略了合成与真实数据间的巨大域差距,导致泛化性能不佳。本文提出IntrinsicReal,一种新型域适应框架,以弥合该域差距。具体而言,IntrinsicReal通过微调IntrinsicAnything,利用新颖的双重伪标签策略选择高质量输出:一是基于分类器预测的绝对置信度阈值进行伪标签,二是基于单个输入物体的分类器预测相对偏好排序进行伪标签。该策略受人类评估启发——识别最佳输出容易,但对次优结果的绝对评分可靠性下降,而相对比较更准确。为此,我们设计了一种两阶段流水线,依次应用这两种伪标签技术,有效将IntrinsicAnything适配至真实域。实验表明,IntrinsicReal在合成和真实数据集上均显著优于现有方法,达到当前最优水平。

原文摘要 · Abstract (English)

Estimating albedo (a.k.a., intrinsic image decomposition) from single RGB images captured in real-world environments (e.g., the MVImgNet dataset) presents a significant challenge due to the absence of paired images and their ground truth albedos. Therefore, while recent methods (e.g., IntrinsicAnything) have achieved breakthroughs by harnessing powerful diffusion priors, they remain predominantly trained on large-scale synthetic datasets (e.g., Objaverse) and applied directly to real-world RGB images, which ignores the large domain gap between synthetic and real-world data and leads to suboptimal generalization performance. In this work, we address this gap by proposing IntrinsicReal, a novel domain adaptation framework that bridges the above-mentioned domain gap for real-world intrinsic image decomposition. Specifically, our IntrinsicReal adapts IntrinsicAnything to the real domain by fine-tuning it using its high-quality output albedos selected by a novel dual pseudo-labeling strategy: i) pseudo-labeling with an absolute confidence threshold on classifier predictions, and ii) pseudo-labeling using the relative preference ranking of classifier predictions for individual input objects. This strategy is inspired by human evaluation, where identifying the highest-quality outputs is straightforward, but absolute scores become less reliable for sub-optimal cases. In these situations, relative comparisons of outputs become more accurate. To implement this, we propose a novel two-phase pipeline that sequentially applies these pseudo-labeling techniques to effectively adapt IntrinsicAnything to the real domain. Experimental results show that our IntrinsicReal significantly outperforms existing methods, achieving state-of-the-art results for albedo estimation on both synthetic and real-world datasets.

图像分解域适应扩散模型反照率估计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。