用可验证的开放问答替代多选题，让视觉语言模型训练更真实可靠。
Beyond Multiple Choice: Verifiable OpenQA for Robust Vision-Language RFT
- 将多选题重写为可验证的开放问题，避免选项泄露信息
- 在20,000个样本上微调后，开放问答准确率提升6个百分点
- 适合追求真实能力评估与高效训练的视觉语言模型研究者
多选题问答（MCQA）常用于评估和强化型微调（RFT）现代多模态语言模型。其受限输出格式便于自动验证，但选项可能泄露可被利用的信号，导致准确率指标不可靠，并在微调中诱导显式或隐式猜题行为。我们提出ReVeL（Rewrite and Verify by LLM），一种将多选题重写为可验证开放问题的框架，根据答案类型分类采用不同重写与验证策略。在微调中，我们将20,000个MCQA样本转换为开放形式，使用GRPO对Qwen2.5-VL模型进行微调。基于ReVeL-OpenQA训练的模型在多选题基准上保持原有准确率，同时开放问答准确率提升约6个百分点，表明数据效率更高、奖励信号更鲁棒。在评估中，ReVeL揭示了多选题基准最高达20个百分点的分数虚高，提升了判断准确率，并降低成本与延迟。代码与数据将公开。
原文摘要 · Abstract (English)
Multiple-choice question answering (MCQA) has been a popular format for evaluating and reinforcement fine-tuning (RFT) of modern multimodal language models. Its constrained output format allows for simplified, deterministic automatic verification. However, we find that the options may leak exploitable signals, which makes the accuracy metrics unreliable for indicating real capabilities and encourages explicit or implicit answer guessing behaviors during RFT. We propose ReVeL (Rewrite and Verify by LLM), a framework that rewrites multiple-choice questions into open-form questions while keeping answers verifiable whenever possible. The framework categorizes questions according to different answer types, apply different rewriting and verification schemes, respectively. When applied for RFT, we converted 20k MCQA examples and use GRPO to finetune Qwen2.5-VL models. Models trained on ReVeL-OpenQA match MCQA accuracy on multiple-choice benchmarks and improve OpenQA accuracy by about six percentage points, indicating better data efficiency and more robust reward signals than MCQA-based training. When used for evaluation, ReVeL also reveals up to 20 percentage points of score inflation in MCQA benchmarks (relative to OpenQA), improves judging accuracy, and reduces both cost and latency. We will release code and data publicly.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。