用语言提示和时序分块提升视频阴影检测精度与速度
DTTNet: Improving Video Shadow Detection via Dark-Aware Guidance and Tokenized Temporal Modeling
- 引入语言匹配模块和暗区语义块，区分阴影与暗物
- 时序分块编码实现高效动态阴影建模，推理速度快
- 适合需要实时视频阴影分析的场景，如自动驾驶
视频阴影检测面临两大挑战：阴影与复杂背景的混淆，以及光照变化下阴影形态的动态演变。为解决阴影-背景歧义，本文提出视觉-语言匹配模块（VMM）与暗区语义块（DSB），利用语言先验提取文本引导特征，显式区分阴影与暗物体。同时引入自适应掩码重加权机制，在训练中降低半影区域权重，并在解码器末尾加入边缘掩码以增强监督。针对时序建模，设计了分词化时序块（TTB），将跨帧阴影语义压缩为可学习的时序标记，实现时空解耦与高效序列编码，计算开销极低。在多个基准数据集上的全面实验表明，该方法达到当前最优性能，并具备实时推理能力。代码已开源。
原文摘要 · Abstract (English)
Video shadow detection confronts two entwined difficulties: distinguishing shadows from complex backgrounds and modeling dynamic shadow deformations under varying illumination. To address shadow-background ambiguity, we leverage linguistic priors through the proposed Vision-language Match Module (VMM) and a Dark-aware Semantic Block (DSB), extracting text-guided features to explicitly differentiate shadows from dark objects. Furthermore, we introduce adaptive mask reweighting to downweight penumbra regions during training and apply edge masks at the final decoder stage for better supervision. For temporal modeling of variable shadow shapes, we propose a Tokenized Temporal Block (TTB) that decouples spatiotemporal learning. TTB summarizes cross-frame shadow semantics into learnable temporal tokens, enabling efficient sequence encoding with minimal computation overhead. Comprehensive Experiments on multiple benchmark datasets demonstrate state-of-the-art accuracy and real-time inference efficiency. Codes are available at https://github.com/city-cheng/DTTNet.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。