利用材质一致的阴影边缘实现测试时自监督,提升去阴影效果
Shadow Removal Refinement via Material-Consistent Shadow Edges
- 通过对比SAM分割与阴影掩码,识别材质一致的阴影边界
- 引入颜色与纹理一致性损失,显著改善野外复杂图像的去阴影效果
- 提供无配对数据的评估新指标与标注数据集,适合算法评测
阴影边界常与材质边界混淆,因二者均表现为亮度或对比度的突变。然而,阴影不会改变表面的固有颜色或纹理。因此,在跨越相同材质区域的阴影边缘两侧,若阴影去除正确,原始颜色和纹理应保持一致。这类阴影/无阴影配对虽极具价值但难以获取。本文核心贡献在于学习识别穿过材质一致区域的阴影边缘,并将其用作测试时的自监督信号以优化去阴影过程。为此,我们微调了图像分割基础模型SAM,生成对阴影不变的分割结果,再通过比较其与阴影掩码,提取材质一致的阴影边缘。基于这些边缘,引入颜色与纹理一致性损失以增强去阴影效果。实验表明,该方法在更具挑战性的野外图像上优于当前最优去阴影方法。此外,我们提出了一种新评估指标和一个标注数据集,可在无需成对阴影/无阴影数据的情况下评估去阴影性能。
原文摘要 · Abstract (English)
Shadow boundaries can be confused with material boundaries as both exhibit sharp changes in luminance or contrast within a scene. However, shadows do not modify the intrinsic color or texture of surfaces. Therefore, on both sides of shadow edges traversing regions with the same material, the original color and textures should be the same if the shadow is removed properly. These shadow/shadow-free pairs are very useful but hard-to-collect supervision signals. The crucial contribution of this paper is to learn how to identify those shadow edges that traverse material-consistent regions and how to use them as self-supervision for shadow removal refinement during test time. To achieve this, we fine-tune SAM, an image segmentation foundation model, to produce a shadow-invariant segmentation and then extract material-consistent shadow edges by comparing the SAM segmentation with the shadow mask. Utilizing these shadow edges, we introduce color and texture-consistency losses to enhance the shadow removal process. We demonstrate the effectiveness of our method in improving shadow removal results on more challenging, in-the-wild images, outperforming the state-of-the-art shadow removal methods. Additionally, we propose a new metric and an annotated dataset for evaluating the performance of shadow removal methods without the need for paired shadow/shadow-free data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。