arXiv:2607.02284cs.CV2026-07中稿 · ECCV

用流匹配实现零样本图像检索的语义精准迁移

FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval

论文配图:FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval
图 1 · 摘自论文原文
  • 将图像检索建模为参考与目标嵌入间的条件语义传输
  • 训练资源仅需传统方法的1/10,且支持否定指令的鲁棒处理
  • 适合追求高效、精准语言控制图像生成的研究者

零样本组合图像检索(ZS-CIR)旨在通过自然语言指令编辑参考图像来检索目标图像,无需领域特定标注三元组。现有方法依赖文本反转将参考图像转为伪文本标记,并在文本空间简单拼接指令,但对细粒度语义易造成损失且脆弱。本文提出新范式FlowCIR,将ZS-CIR视为参考与目标嵌入间的条件语义传输。利用条件流匹配,模型学习一个轻量级传输场,将指令表示映射至与目标对齐的查询嵌入,条件于参考图像。由于仅在预提取的视觉-语言模型(VLM)嵌入上操作,且仅训练小型传输模块而不更新图像或文本编码器,该方法计算效率更高,所需训练资源约为传统文本反转方法的1/10。进一步发现,否定和移除是当前基于VLM方法的主要失败模式。为此,提出仅推理阶段的多否定引导策略,使含否定的相对指令避开被否定语义,缓解VLM在否定处理上的局限,提升在高否定查询下的鲁棒性。在标准CIR基准上的大量实验表明,FlowCIR性能强且具有竞争力。

原文摘要 · Abstract (English)

Zero-shot composed image retrieval (ZS-CIR) aims to retrieve a target image by editing a reference image with a natural-language instruction, without relying on domain-specific annotated triplets. Most existing ZS-CIR methods rely on textual inversion to translate the reference image into pseudo-text tokens and then compose them with the instruction via simple concatenation in the text space, which can be lossy and brittle for fine-grained semantics. In this work, we propose a new paradigm, namely FlowCIR, that casts ZS-CIR as conditional semantic transport between reference and target embeddings. Leveraging \emph{conditional flow matching}, our model learns a lightweight transport field that maps the instruction representation toward a target-aligned query embedding conditioned on the reference image. Since FlowCIR operates on pre-extracted VLM embeddings and trains only a small transport module without updating the image or text encoder, it offers a computationally efficient training protocol compared with prior textual-inversion-based approaches. The resulting framework is training-efficient, requiring roughly $10\times$ fewer training resources than prior textual-inversion-based approaches. We further identify negation and removal as a major failure mode of VLM-based composition. To address this, we propose an inference-only Multi-Negative Steering strategy that steers a negation-containing relative instruction away from its negated semantics, mitigating the limited negation handling of VLMs and improving robustness on negation-heavy queries. Extensive experiments on standard CIR benchmarks demonstrate that FlowCIR achieves strong and competitive performance compared with recent ZS-CIR methods.

图像检索流匹配零样本语言控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。