用语言语义辅助识别阴影,解决暗区混淆难题
Revisiting Shadow Detection from a Vision-Language Perspective

- 引入视觉-语言对齐,用文本描述增强阴影判断
- 在多个基准上准确率超传统方法,尤其在模糊场景表现更优
- 轻量级设计仅需不到1%参数可训练,适合部署
阴影检测通常被建模为依赖像素级视觉监督的密集预测任务,但在视觉模糊情况下(如阴影与固有暗面外观相似),仅靠视觉线索难以建立稳定判别规则。本文从视觉-语言视角重新审视该问题,提出SVL框架,利用语言作为显式语义参考来区分视觉上相似的暗区。SVL通过场景级阴影比例回归对齐图像全局表示与阴影相关文本嵌入,并通过全局到局部耦合及局部块级约束将语义指导传递至密集预测。基于冻结的DINOv3图像编码器,仅训练轻量级投影与解码模块,参数效率高达不足1%。在多个阴影检测基准上的实验表明,该方法整体性能优异,且在视觉模糊条件下鲁棒性显著提升。代码已开源。
原文摘要 · Abstract (English)
Shadow detection is commonly formulated as a vision-driven dense prediction problem, where models rely primarily on pixel-wise visual supervision to distinguish shadows from non-shadow regions. However, this formulation can become unreliable in visually ambiguous cases, where similar dark regions may correspond either to cast shadows or to intrinsically dark surfaces, making visual evidence alone insufficient for establishing a stable decision rule. In this work, we revisit shadow detection from a vision--language perspective and argue that robust prediction benefits from an explicit semantic reference beyond visual cues alone. We propose SVL, a Shadow Vision--Language framework that uses language as an explicit semantic reference to disambiguate shadows from visually similar dark regions. SVL aligns global image representations with shadow-related text embeddings through scene-level shadow ratio regression, and transfers this semantic guidance to dense prediction via global-to-local coupling and local patch-level constraints. Built on a frozen DINOv3 image encoder, SVL learns only lightweight projection and decoding modules, yielding a parameter-efficient design with less than $1\%$ trainable parameters. Extensive experiments on multiple shadow detection benchmarks, including dedicated hard-case evaluations, suggest strong overall performance and improved robustness under visually ambiguous conditions. Code is available at https://github.com/harrytea/SVL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。