arXiv:2602.08603cs.AI2026-02

用优化方法提升图像检索的智能规划能力,让系统更准更快。

OSCAR: Optimization-Steered Agentic Planning for Composed Image Retrieval

  • 将复杂检索任务转为可计算的优化问题,生成最优推理路径
  • 仅用10%训练数据就超越顶尖模型,泛化能力强
  • 适合需要高精度多条件图像检索的工业应用

组合图像检索(CIR)需对异构视觉与文本约束进行复杂推理。现有方法主要分为统一嵌入检索(受限于单一模型视角)和启发式代理检索(受制于低效试错协调)。为此,我们提出OSCAR,一种面向组合图像检索的优化引导代理规划框架。首次将代理式CIR从启发式搜索重构为严谨的轨迹优化问题。采用离线-在线双阶段范式:离线阶段将检索拆解为原子选择与组合,建模为两阶段混合整数规划问题,通过严格的布尔集合运算,数学推导出最大化真实标签覆盖的最优路径,并存入“黄金库”作为上下文示范;在线阶段,以该库引导视觉语言模型(VLM)规划器进行推理。在三个公开基准及一个私有工业基准上实验证明,OSCAR持续优于当前最优基线,尤其在仅使用10%训练数据时仍表现优异,体现规划逻辑强泛化性而非数据记忆。

原文摘要 · Abstract (English)

Composed image retrieval (CIR) requires complex reasoning over heterogeneous visual and textual constraints. Existing approaches largely fall into two paradigms: unified embedding retrieval, which suffers from single-model myopia, and heuristic agentic retrieval, which is limited by suboptimal, trial-and-error orchestration. To this end, we propose OSCAR, an optimization-steered agentic planning framework for composed image retrieval. We are the first to reformulate agentic CIR from a heuristic search process into a principled trajectory optimization problem. Instead of relying on heuristic trial-and-error exploration, OSCAR employs a novel offline-online paradigm. In the offline phase, we model CIR via atomic retrieval selection and composition as a two-stage mixed-integer programming problem, mathematically deriving optimal trajectories that maximize ground-truth coverage for training samples via rigorous boolean set operations. These trajectories are then stored in a golden library to serve as in-context demonstrations for online steering of VLM planner at online inference time. Extensive experiments on three public benchmarks and a private industrial benchmark show that OSCAR consistently outperforms SOTA baselines. Notably, it achieves superior performance using only 10% of training data, demonstrating strong generalization of planning logic rather than dataset-specific memorization.

图像检索智能规划优化方法VLM

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。