用多智能体协作实现跨模态图像检索，精准匹配图文组合查询
XR: Cross-Modal Agents for Composed Image Retrieval
- 三类智能体分工：生成目标表征、粗筛相似项、验证事实一致性
- 在FashionIQ等数据集上比基线提升38%，无需训练即可达到顶尖性能
- 适合需要高精度跨模态理解的检索场景，如时尚、设计领域
检索正被代理式AI重新定义，要求超越传统相似性匹配的多模态推理。组合图像检索（CIR）体现了这一转变：每个查询结合参考图像与文本修改，需跨模态的组合理解。尽管基于嵌入的CIR方法已取得进展，但其视角狭窄，仅捕捉有限的跨模态线索且缺乏语义推理。为此，我们提出XR——一种免训练的多智能体框架，将检索重构为渐进式协同推理过程。该框架协调三种专用智能体：想象智能体通过跨模态生成合成目标表征，相似性智能体通过混合匹配进行粗筛选，问题智能体通过定向推理进行细筛选以验证事实一致性。通过渐进式多智能体协作，XR迭代优化检索结果，满足语义与视觉双重约束，在FashionIQ、CIRR和CIRCO数据集上相比强训练自由及训练基线最高提升38%，消融实验表明每类智能体均不可或缺。代码已公开：https://01yzzyu.github.io/xr.github.io/
原文摘要 · Abstract (English)
Retrieval is being redefined by agentic AI, demanding multimodal reasoning beyond conventional similarity-based paradigms. Composed Image Retrieval (CIR) exemplifies this shift as each query combines a reference image with textual modifications, requiring compositional understanding across modalities. While embedding-based CIR methods have achieved progress, they remain narrow in perspective, capturing limited cross-modal cues and lacking semantic reasoning. To address these limitations, we introduce XR, a training-free multi-agent framework that reframes retrieval as a progressively coordinated reasoning process. It orchestrates three specialized types of agents: imagination agents synthesize target representations through cross-modal generation, similarity agents perform coarse filtering via hybrid matching, and question agents verify factual consistency through targeted reasoning for fine filtering. Through progressive multi-agent coordination, XR iteratively refines retrieval to meet both semantic and visual query constraints, achieving up to a 38% gain over strong training-free and training-based baselines on FashionIQ, CIRR, and CIRCO, while ablations show each agent is essential. Code is available: https://01yzzyu.github.io/xr.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。