用校准机制解决图像组合检索中的模糊问题,提升查询准确性。
Resolving Ambiguity in Composed Image Retrieval via Calibrated Interaction

- 引入置信区间层,根据不确定性动态生成候选集大小来衡量模糊程度。
- 在多个基准上实现与单轮顶尖方法相当的召回率,交互次数减少超70%。
- 首个报告任务覆盖性和校准性的方法,适合需要对话澄清的复杂检索场景。
组合图像检索(CIR)通过参考图像和文本描述来搜索图像库。尽管从三元组训练到零样本及生成式方法快速进展,现有系统均假设查询对应唯一目标,并以 Recall@K 评分。我们指出这与任务本质相悖:如“更正式”这类查询并非指定具体图像,而是描述一个区域,用户意图本就不确定。此不确定性是误负结果的根本原因,使模型无法区分精确与模糊查询。本文将CIR重构为不确定下的校准意图解析:在检索器外增加共形预测层,返回具有覆盖率保证的候选集,其大小即为模糊度的合理度量;当集合过大时,采用期望信息增益策略提出最有效的澄清问题(来自可解释的模糊轴),从而缩小候选集。我们提出 AmbiCIR 基准与人工验证的用户模拟器,重新激活了已废弃的辅助与对话标注,并扩展了 CIRCO 的多正例设置。在开放域与时尚基准上,该方法在单轮表现上媲美当前最优,同时在远低于传统对话基线的交互预算内达成目标,且首次为该任务报告有效覆盖率与校准性。
原文摘要 · Abstract (English)
Composed image retrieval (CIR) searches a corpus with a reference image and a text describing how to modify it. Despite rapid progress from triplet-trained compositors to zero-shot and generative methods, essentially all systems share one assumption: that a query maps to a single target, scored by Recall@K against one annotation. We argue this is fundamentally at odds with the task. A query such as make it more formal does not name an image but a region of the corpus, and which member the user intends is genuinely underdetermined. This underspecification is the root of the well-known false-negative problem and leaves current models unable to tell a precise query from an ambiguous one. We reframe CIR as calibrated intent resolution under uncertainty: a retriever is wrapped in a conformal prediction layer that returns a candidate set with a coverage guarantee and whose size is a principled measure of ambiguity; when the set is large, an expected-information-gain policy asks the single most useful clarifying question, drawn from interpretable ambiguity axes, and the set contracts. We introduce AmbiCIR, a benchmark and human-validated user simulator that revive the dormant auxiliary and dialogue annotations of CIRR and extend the multiple-positive setting of CIRCO. Across open-domain and fashion benchmarks our method matches single-turn state of the art, confirming calibrated resolution is cost-free on precise queries, while reaching the intended target in a fraction of the interaction budget required by naive conversational baselines, and it is the first to report valid coverage and calibration for the task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。