不训练即可零样本图像检索，精准区分要保留、改变、删除的细节。
MulVec: Fine-Grained Role-Aware Matching for Training-Free Zero-Shot Composed Image Retrieval
- 将查询拆解为全局、期望、保留、禁止四类角色，实现细粒度匹配
- 在CIRCO上相比最强方法提升23.0% mAP@5，其他数据集也最优
- 无需训练，适配多场景，适合快速部署的图像检索应用
训练无关的零样本组合图像检索可在无任务特定三元组学习的情况下，从参考图与文本编辑中定位目标图像。现有方法通常将目标整体描述并匹配全局图像表征，易混淆不同语义线索并丢失细粒度信息。本文提出MULVEC，其编译器生成结构化查询记录，映射至四类检索角色：全局（描述完整目标）、期望（应出现内容）、保留（应保持内容）、禁止（应消失内容）。冻结编码器将查询映射为单一目标描述向量和角色特异性探针向量，每个候选则由全局视觉向量与局部视觉向量库表示。各角色基于共享证据执行任务，固定加权和评分完成单次遍历排序。在CIRCO、CIRR、FashionIQ三个数据集及三种主干网络尺度下，MULVEC相较最强对比方法在CIRCO上提升最大23.0% mAP@5，且在其余数据集表现最佳。
原文摘要 · Abstract (English)
Training-free zero-shot composed image retrieval finds a target image in a gallery from a reference image and a text edit without learning from task-specific image triplets. Existing methods typically describe the target as a whole and match this description with a global image representation. This global matching can mix different semantic cues and lose fine- grained details. We propose MULVEC, a role-aware method whose compiler produces a structured query record that is mapped to four retrieval roles: Global describes the full target, Desired states what should appear, Preserve states what should remain, and Forbidden states what should disappear. Frozen encoders map the query to one target description vector and role-specific probe vectors, while each candidate is represented by one global visual vector and a bank of local visual vectors. The retrieval roles then use this shared evidence for their respective purposes, and a fixed weighted sum of their scores ranks the entire gallery in a single retrieval pass. Across CIRCO, CIRR, and FashionIQ and three backbone scales, MULVEC improves CIRCO mAP@5 by up to 23.0% over the strongest compared method and gives the best CIRR and FashionIQ results in our comparison.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。