arXiv:2512.19554cs.LGcs.AI2025-12被引 6

让错误成为训练信号,提升多模态推理的准确率与稳定性。

CARE What Fails: Contrastive Anchored-REflection for Verifiable Multimodal Reasoning

  • 以失败样本为核心,构建对比学习框架,强化错误信息利用。
  • 在六个视觉推理基准上,使Qwen2.5-VL-7B准确率提升4.6点。
  • 无需测试时推理,即可将近似正确结果转化为有效正样本。

组相对强化学习结合可验证奖励(RLVR)常浪费已有失败数据。当所有轨迹均错误时梯度停滞;当某条偶然正确时,更新通常忽略其他接近但错误的路径,导致信用分配错误。我们提出CARE(对比锚定反射),一种以失败为中心的多模态推理后训练框架,将错误转化为监督信号。CARE结合:(i) 锚定对比目标,在最优轨迹周围形成紧凑子组,并包含语义相近的困难负样本,进行仅负样本缩放的组内z-score归一化,引入全负样本救援机制防止零信号批次;(ii) 反思引导重采样(RGR),一次结构化自修复,重写代表性失败轨迹并用相同验证器重新评分,无需测试时反思即可将近似正确结果转为可用正样本。CARE在提升准确率和训练平滑性的同时,明确增加了来自失败的训练信号占比。在Qwen2.5-VL-7B上,相较GRPO在六个可验证视觉推理基准上平均准确率提升4.6点;使用Qwen3-VL-8B时,在MathVista和MMMU-Pro上达到竞争性或领先水平,评估协议一致。

原文摘要 · Abstract (English)

Group-relative reinforcement learning with verifiable rewards (RLVR) often wastes the most informative data it already has the failures. When all rollouts are wrong, gradients stall; when one happens to be correct, the update usually ignores why the others are close-but-wrong, and credit can be misassigned to spurious chains. We present CARE (Contrastive Anchored REflection), a failure-centric post-training framework for multimodal reasoning that turns errors into supervision. CARE combines: (i) an anchored-contrastive objective that forms a compact subgroup around the best rollout and a set of semantically proximate hard negatives, performs within-subgroup z-score normalization with negative-only scaling, and includes an all-negative rescue to prevent zero-signal batches; and (ii) Reflection-Guided Resampling (RGR), a one-shot structured self-repair that rewrites a representative failure and re-scores it with the same verifier, converting near-misses into usable positives without any test-time reflection. CARE improves accuracy and training smoothness while explicitly increasing the share of learning signal that comes from failures. On Qwen2.5-VL-7B, CARE lifts macro-averaged accuracy by 4.6 points over GRPO across six verifiable visual-reasoning benchmarks; with Qwen3-VL-8B it reaches competitive or state-of-the-art results on MathVista and MMMU-Pro under an identical evaluation protocol.

多模态推理强化学习错误学习自修复

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。