用物理约束提升阴影去除效果,兼顾局部纹理与全局光照一致性。
CFSR: Geometry-Conditioned Shadow Removal via Physical Disentanglement

- 引入3D几何信息与大模型语义,构建物理可解释的修复框架。
- 在多个基准上达到当前最优性能,显著改善严重退化区域恢复效果。
- 适合关注图像修复、计算机视觉物理建模的研究者和开发者。
传统阴影去除网络常将图像修复视为无约束映射,缺乏平衡局部纹理恢复与全局光照一致性的物理可解释性。为此,我们提出CFSR,一种多模态先验驱动的框架,将阴影去除重定义为物理约束下的修复过程。通过融合3D几何线索与大规模基础模型语义,CFSR有效弥合2D-3D域差距。首先,将观测值映射至自定义的HVI色彩空间,抑制阴影诱发噪声,并鲁棒融合RGB数据与估计深度先验。核心在于几何与语义双重显式引导注意力机制,利用DINO特征与3D表面法线直接调节注意力亲和矩阵,结构化施加物理光照约束。为恢复严重退化区域,通过冻结的CLIP编码器注入整体先验。最后,频率协同重建模块(FCRM)通过解码过程解耦,实现最优合成。在几何先验条件下,FCRM无缝协调高频频段遮挡边界锐化与低频段全局光照恢复。大量实验表明,CFSR在多个挑战性基准上均达当前最优性能。
原文摘要 · Abstract (English)
Traditional shadow removal networks often treat image restoration as an unconstrained mapping, lacking the physical interpretability required to balance localized texture recovery with global illumination consistency. To address this, we propose CFSR, a multi-modal prior-driven framework that reframes shadow removal as a physics-constrained restoration process. By seamlessly integrating 3D geometric cues with large-scale foundation model semantics, CFSR effectively bridges the 2D-3D domain gap. Specifically, we first map observations into a custom HVI color space to suppress shadow-induced noise and robustly fuse RGB data with estimated depth priors. At its core, our Geometric & Semantic Dual Explicit Guided Attention mechanism utilizes DINO features and 3D surface normals to directly modulate the attention affinity matrix, structurally enforcing physical lighting constraints. To recover severely degraded regions, we inject holistic priors via a frozen CLIP encoder. Finally, our Frequency Collaborative Reconstruction Module (FCRM) achieves an optimal synthesis by decoupling the decoding process. Conditioned on geometric priors, FCRM seamlessly harmonizes the reconstruction of sharp high-frequency occlusion boundaries with the restoration of low-frequency global illumination. Extensive experiments demonstrate that CFSR achieves state-of-the-art performance across multiple challenging benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。