用真实图像替代噪声，直接预测室内场景的确定性属性。
DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Rendering
- 以源图像为输入，通过流匹配直接预测内在属性
- 在合成与真实数据集上均显著优于现有方法
- 适合需要高保真逆渲染的场景重建任务
近期方法表明，可通过微调预训练扩散模型，学习图像条件下的噪声到内在属性映射，实现生成式逆渲染。然而，这类方法在鲁棒性与高质量输出方面仍存在瓶颈，因其依赖结构和外观退化的噪声图像进行内在属性预测，而这些信息对逆渲染至关重要。为此，我们提出DNF-Intrinsic，一种从预训练扩散模型微调而来的稳健高效逆渲染方法。该方法采用源图像而非高斯噪声作为输入，通过流匹配直接预测确定性的内在属性，并设计生成式渲染器约束预测结果在物理上与源图像一致。在合成与真实世界数据集上的实验表明，本方法明显优于现有最先进方法。
原文摘要 · Abstract (English)
Recent methods have shown that pre-trained diffusion models can be fine-tuned to enable generative inverse rendering by learning image-conditioned noise-to-intrinsic mapping. Despite their remarkable progress, they struggle to robustly produce high-quality results as the noise-to-intrinsic paradigm essentially utilizes noisy images with deteriorated structure and appearance for intrinsic prediction, while it is common knowledge that structure and appearance information in an image are crucial for inverse rendering. To address this issue, we present DNF-Intrinsic, a robust yet efficient inverse rendering approach fine-tuned from a pre-trained diffusion model, where we propose to take the source image rather than Gaussian noise as input to directly predict deterministic intrinsic properties via flow matching. Moreover, we design a generative renderer to constrain that the predicted intrinsic properties are physically faithful to the source image. Experiments on both synthetic and real-world datasets show that our method clearly outperforms existing state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。