用评分表机制提升视觉语言模型回答的视觉一致性。
V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning

- 将答案拆解为原子命题,按视觉一致、推理一致等维度评分
- 在5万样本数据集上训练,显著提升视觉相关任务表现
- 适合需要高视觉可信度的多模态应用开发者
视觉语言模型常生成看似合理但缺乏视觉依据的回答:一个无关对象或错误图表值即可破坏整体可信度。我们认为这是多模态后训练中信用分配失败所致。标量奖励仅判断答案是否可接受,却无法定位哪些视觉事实被支持、哪些推理步骤有效或指令约束是否遵守。为此提出基于评分表的强化学习(V-Rubrics),将参考答案分解为原子命题,从视觉忠实性(VF)、推理一致性(RC)和指令遵循性(IF)三方面打分。该评分体系提供结构化部分得分,并在有支撑证据时精确定位信用归属。我们首先在公开的OpenMMReasoner-SFT-874K语料上微调Qwen3-VL-8B-Instruct获得SFT检查点,再从17个视觉基准来源构建包含50,248个样本的V-Rubrics 50K数据集,通过规则过滤、拒绝采样确定难度,并用Gemini-3-Pro按统一结构化提示进行标注。基于相同SFT检查点,采用组件级、前缀局部化的评分信用进行训练。实验表明,基于评分表的GRPO优于共享SFT基线和仅答案奖励的GRPO,尤其在知识导向与视觉推理基准上提升显著。结果证明评分表是视觉后训练中有效的奖励抽象工具。
原文摘要 · Abstract (English)
Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。