用扩散模型去除阴影，还能保留细节。
Detail-Preserving Latent Diffusion for Stable Shadow Removal
- 分两阶段微调扩散模型，先去阴影再注入细节。
- 跨数据集测试表现更好，泛化能力更强。
- 适合需要高质量去阴影的图像修复场景。
在复杂全局光照场景中实现高质量且泛化能力强的阴影去除极具挑战性。由于阴影去除数据集多样性有限，现有方法易过拟合训练数据，导致在未见场景下性能下降。为此，我们利用预训练稳定扩散（Stable Diffusion, SD）模型丰富的视觉先验，提出一种两阶段微调流程，以实现稳定高效的阴影去除。第一阶段固定变分自编码器（VAE），在潜在空间微调去噪器，可有效去除阴影但可能损失高频细节。为解决此问题，我们引入第二阶段——细节注入阶段，该阶段从VAE编码器中选择性提取特征，调制解码器，将精细细节注入最终结果。实验表明，该方法优于当前最优去阴影技术。跨数据集评估进一步证明其对未见数据具有良好的泛化能力，提升了去阴影方法的实际应用价值。
原文摘要 · Abstract (English)
Achieving high-quality shadow removal with strong generalizability is challenging in scenes with complex global illumination. Due to the limited diversity in shadow removal datasets, current methods are prone to overfitting training data, often leading to reduced performance on unseen cases. To address this, we leverage the rich visual priors of a pre-trained Stable Diffusion (SD) model and propose a two-stage fine-tuning pipeline to adapt the SD model for stable and efficient shadow removal. In the first stage, we fix the VAE and fine-tune the denoiser in latent space, which yields substantial shadow removal but may lose some high-frequency details. To resolve this, we introduce a second stage, called the detail injection stage. This stage selectively extracts features from the VAE encoder to modulate the decoder, injecting fine details into the final results. Experimental results show that our method outperforms state-of-the-art shadow removal techniques. The cross-dataset evaluation further demonstrates that our method generalizes effectively to unseen data, enhancing the applicability of shadow removal methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。