解决图像检索中的噪声标注问题，提升复杂查询的精准度。
HABIT: Chrono-Synergia Robust Progressive Learning Framework for Composed Image Retrieval
- 通过互信息变化率识别干净样本，定位真实语义修改
- 双一致性渐进学习机制，抗噪声下仍保持高精度检索
- 适合有噪声数据的个性化搜索与推荐系统应用
组合图像检索（CIR）是一种灵活的图像检索范式，用户可通过参考图像与修改文本的多模态查询精准定位目标图像。尽管该任务在个性化搜索与推荐系统中展现出广阔前景，但在实际场景中面临严重挑战——噪声三元组对应（NTC）问题，主要源于三元组标注成本高且具有主观性。为此，本文识别出两大核心挑战：精确估计组合语义差异，以及对修改语义差异的不足渐进适应。为此提出一种时间协同鲁棒渐进学习框架HABIT，包含两个核心模块：首先，互知识评估模块通过计算组合特征与目标图像间互信息的转移率，量化样本纯净度，有效识别符合预期修改语义的干净样本；其次，双一致性渐进学习模块引入历史与当前模型间的协同机制，模拟人类习惯形成过程，保留良好习惯、校正不良习惯，从而在存在NTC的情况下实现稳健学习。在两个标准CIR数据集上的大量实验表明，HABIT在不同噪声比例下显著优于多数现有方法，展现出卓越的鲁棒性与检索性能。代码已公开于https://github.com/Lee-zixu/HABIT。
原文摘要 · Abstract (English)
Composed Image Retrieval (CIR) is a flexible image retrieval paradigm that enables users to accurately locate the target image through a multimodal query composed of a reference image and modification text. Although this task has demonstrated promising applications in personalized search and recommendation systems, it encounters a severe challenge in practical scenarios known as the Noise Triplet Correspondence (NTC) problem. This issue primarily arises from the high cost and subjectivity involved in annotating triplet data. To address this problem, we identify two central challenges: the precise estimation of composed semantic discrepancy and the insufficient progressive adaptation to modification discrepancy. To tackle these challenges, we propose a cHrono-synergiA roBust progressIve learning framework for composed image reTrieval (HABIT), which consists of two core modules. First, the Mutual Knowledge Estimation Module quantifies sample cleanliness by calculating the Transition Rate of mutual information between the composed feature and the target image, thereby effectively identifying clean samples that align with the intended modification semantics. Second, the Dual-consistency Progressive Learning Module introduces a collaborative mechanism between the historical and current models, simulating human habit formation to retain good habits and calibrate bad habits, ultimately enabling robust learning under the presence of NTC. Extensive experiments conducted on two standard CIR datasets demonstrate that HABIT significantly outperforms most methods under various noise ratios, exhibiting superior robustness and retrieval performance. Codes are available at https://github.com/Lee-zixu/HABIT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。