用强化学习选视觉示范，让大模型更准做客观推理
Learning to Select Visual In-Context Demonstrations
- 用强化学习构建示范选择策略，兼顾相关性与多样性
- 在5个视觉回归任务中，新方法显著优于传统最近邻法
- 适合需要精准事实推理的视觉任务，如测量、估价等
多模态大语言模型通过上下文学习适应视觉任务，其效果高度依赖示范质量。当前主流的无监督k近邻（kNN）搜索策略虽简单，但在复杂事实回归任务中表现不佳，因会选取冗余示例，无法覆盖完整输出范围。本文将示范选择重构为序列决策问题，提出学习示范选择（LSD）框架，训练一个基于双分支DQN与查询中心Transformer解码器的强化学习代理，以最大化下游任务性能。在五个视觉回归基准上评估发现：尽管kNN在主观偏好任务中仍最优，但LSD在客观事实回归任务中显著超越基线。通过平衡视觉相关性与多样性，LSD能更好界定回归边界，揭示了在视觉上下文学习中，为何必须采用学习型选择机制。
原文摘要 · Abstract (English)
Multimodal Large Language Models (MLLMs) adapt to visual tasks via in-context learning (ICL), which relies heavily on demonstration quality. The dominant demonstration selection strategy is unsupervised k-Nearest Neighbor (kNN) search. While simple, this similarity-first approach is sub-optimal for complex factual regression tasks; it selects redundant examples that fail to capture the task's full output range. We reframe selection as a sequential decision-making problem and introduce Learning to Select Demonstrations (LSD), training a Reinforcement Learning agent to construct optimal demonstration sets. Using a Dueling DQN with a query-centric Transformer Decoder, our agent learns a policy that maximizes MLLM downstream performance. Evaluating across five visual regression benchmarks, we uncover a crucial dichotomy: while kNN remains optimal for subjective preference tasks, LSD significantly outperforms baselines on objective, factual regression tasks. By balancing visual relevance with diversity, LSD better defines regression boundaries, illuminating when learned selection is strictly necessary for visual ICL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。