解决文本检索中关键词被淹没的问题，提升图像匹配精度。
CalibCLIP: Contextual Calibration of Dominant Semantics for Text-Driven Image Retrieval
- 通过分离视觉特征区域，动态抑制主导词的干扰
- 在文本空间区分通用与关键概念，增强差异表达
- 无需训练，在7个基准上均实现稳定性能提升
现有视觉语言模型存在结构缺陷：少数贡献度低的词可能过度主导全局语义，抑制文本驱动图像检索中的判别性特征。为此，我们提出无需训练的CalibCLIP方法，用于校准主导词的抑制效应。在视觉空间，设计对比视觉增强器（CVE），将视觉特征解耦为目标区域与低信息区域，识别主导词并动态压制其表征；在文本空间，引入判别概念校准器（DCC），区分文本查询中的通用概念与判别性概念，缓解通用概念带来的挑战，强化判别性概念的表示，从而增强相似样本间的区分能力。大量实验表明，该方法在三个图像检索任务的七个基准上均取得一致提升，验证了其有效性。代码已公开于https://github.com/kangbin98/CalibCLIP。
原文摘要 · Abstract (English)
Existing Visual Language Models (VLMs) suffer structural limitations where a few low contribution tokens may excessively capture global semantics, dominating the information aggregation process and suppressing the discriminative features in text-driven image retrieval tasks. To address this, we introduce \textbf{CalibCLIP}, a training-free method designed to calibrate the suppressive effect of dominant tokens. Specifically, in the visual space, we propose the Contrastive Visual Enhancer (CVE), which decouples visual features into target and low information regions. Subsequently, it identifies dominant tokens and dynamically suppresses their representations.In the textual space, we introduce the Discriminative Concept Calibrator (DCC), which aims to differentiate between general and discriminative concepts within the text query. By mitigating the challenges posed by generic concepts and improving the representations of discriminative concepts, DCC strengthens the differentiation among similar samples. Finally, extensive experiments demonstrate consistent improvements across seven benchmarks spanning three image retrieval tasks, underscoring the effectiveness of CalibCLIP. Code is available at: https://github.com/kangbin98/CalibCLIP
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。