用语言提示增强雾霾中物体语义，提升检测效果
Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environments
- 用语言提示替代图像增强，通过CLIP引导交叉熵损失
- 在HazyCOCO数据集上实现新最优性能，准确率显著提升
- 适合做雾霾环境下目标检测的研究者与工程应用
雾霾环境中物体检测面临挑战，因物体退化导致几乎不可见，语义被环境噪声削弱，使检测器难以识别。传统方法依赖图像增强以恢复弱化语义，但增强模块存在不稳定性。本文提出新方案：不通过图像增强，而是利用语言提示来增强弱化语义。设计近似互斥性（AME）为交叉熵损失提供可信权重，形成CLIP引导的交叉熵损失（CLIP-CE），该权重评估物体语义削弱程度。通过CLIP-CE反向传播，弱化语义得以增强，使退化物体更易检测。此外，提出自适应微调的AME（FAME），根据预测置信度动态调整权重，弥补AME优化不平衡问题。构建大规模合成雾霾数据集HazyCOCO，含61258张图像。实验表明，所提方法达到当前最优性能。代码与数据集将公开。
原文摘要 · Abstract (English)
Object detection in hazy environments is challenging because degraded objects are nearly invisible and their semantics are weakened by environmental noise, making it difficult for detectors to identify. Common approaches involve image enhancement to boost weakened semantics, but these methods are limited by the instability of enhanced modules. This paper proposes a novel solution by employing language prompts to enhance weakened semantics without image enhancement. Specifically, we design Approximation of Mutual Exclusion (AME) to provide credible weights for Cross-Entropy Loss, resulting in CLIP-guided Cross-Entropy Loss (CLIP-CE). The provided weights assess the semantic weakening of objects. Through the backpropagation of CLIP-CE, weakened semantics are enhanced, making degraded objects easier to detect. In addition, we present Fine-tuned AME (FAME) which adaptively fine-tunes the weight of AME based on the predicted confidence. The proposed FAME compensates for the imbalanced optimization in AME. Furthermore, we present HazyCOCO, a large-scale synthetic hazy dataset comprising 61258 images. Experimental results demonstrate that our method achieves state-of-the-art performance. The code and dataset will be released.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。