无需训练即可去除阴影，利用光照迁移保持图像真实感
FreeShadow: Training-Free Shadow Removal via Illumination Transfer and Selective Content Preservation in Diffusion Models
- 基于预训练扩散模型，通过光照迁移注意力机制修复光照
- 在多个数据集上实现强泛化，无须微调且生成图像自然逼真
- 适合快速部署于真实场景，尤其适用于无标注数据的图像修复
现有监督与无监督阴影去除方法因训练数据多样性不足而泛化能力有限，零样本方法则常产生伪影并需耗时的测试阶段优化。为此，我们提出FreeShadow，一种基于预训练扩散模型的训练自由阴影去除方法，利用扩散先验实现无需训练或优化的阴影消除。针对光照恢复，提出光照迁移注意力（ITA），重加权扩散模型中的自注意力图，将非阴影区域的光照信息迁移到阴影区域。针对内容保真，分析光照变化对自注意力图和潜在高频特征的影响，选择性保留光照不变成分，以维持内容完整性并抑制残余阴影。进一步提出局部纹理保真重光照（LTPR）以缓解VAE压缩引起的局部纹理错位。大量实验表明，该方法具备强泛化能力，生成图像真实自然。
原文摘要 · Abstract (English)
Existing supervised and unsupervised shadow removal methods often suffer from limited generalization due to the insufficient diversity of available training datasets, while zero-shot methods tend to produce artifacts and require time-consuming test-time optimization. To address these issues, we propose FreeShadow, a training-free shadow removal method built upon pretrained diffusion models, which exploits diffusion priors for shadow removal without any training or optimization. For illumination recovery, we propose an illumination transfer attention (ITA), which re-weights the self-attention maps in diffusion model to transfer illumination cues from non-shadow to shadow regions. For content preservation, we analyze the effects of illumination variations on self-attention maps and latent high-frequency features in diffusion model, and selectively preserve illumination-invariant components to maintain content fidelity while suppressing residual shadows. We further propose local texture-preserving relighting (LTPR) to mitigate local texture misalignment caused by VAE compression. Extensive experiments demonstrate that our method achieves strong generalization and produces realistic shadow-free images.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。