让视觉语言模型学会一次说清关键细节，减少推理时的追问。
Clarification as Supervision: Reinforcement Learning for Vision-Language Interfaces
- 通过强化学习让模型主动补全推理所需细节，避免后续追问。
- 在7个数学视觉推理任务上平均准确率提升4.4分。
- 无需人工标注，仅靠交互就能优化模型输出质量。
近期纯文本模型展现出强大的数学推理能力。将其拓展至视觉领域需依赖视觉语言模型将图像转化为文本描述。然而，当前模型为人类读者生成描述，常遗漏推理系统所需的精确信息，导致接口错配：推理失败往往并非因推理能力不足，而是缺乏关键视觉信息。本文提出自适应澄清强化学习（AC-RL），通过训练中产生的澄清请求识别信息缺口；通过惩罚需澄清的成功案例，促使模型在首次输出时提供完整信息，使推理系统可单次完成任务。AC-RL在七个视觉数学推理基准上相比预训练基线平均准确率提升4.4点，分析显示若允许澄清，可减少最多39%的请求次数。通过将澄清视为隐式监督信号，证明了仅通过交互即可有效学习视觉语言接口，无需显式标注。
原文摘要 · Abstract (English)
Recent text-only models demonstrate remarkable mathematical reasoning capabilities. Extending these to visual domains requires vision-language models to translate images into text descriptions. However, current models, trained to produce captions for human readers, often omit the precise details that reasoning systems require. This creates an interface mismatch: reasoners often fail not due to reasoning limitations but because they lack access to critical visual information. We propose Adaptive-Clarification Reinforcement Learning (AC-RL), which teaches vision models what information reasoners need through interaction. Our key insight is that clarification requests during training reveal information gaps; by penalizing success that requires clarification, we create pressure for comprehensive initial captions that enable the reasoner to solve the problem in a single pass. AC-RL improves average accuracy by 4.4 points over pretrained baselines across seven visual mathematical reasoning benchmarks, and analysis shows it would cut clarification requests by up to 39% if those were allowed. By treating clarification as a form of implicit supervision, AC-RL demonstrates that vision-language interfaces can be effectively learned through interaction alone, without requiring explicit annotations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。