通过可验证的进化机制,提升多模态数学推理的数据质量与规模。
VeriEvol: Scaling Multimodal Mathematical Reasoning via Verifiable Evol-Instruct

- 用分步进化生成更难且图像相关的题目,提升数据难度。
- 引入多源反例验证器,确保答案可靠性,准确率提升至54.73%。
- 适合追求高质量视觉数学推理数据的研究者与开发者使用。
在扩展视觉数学推理的强化学习时,仅生成更难问题不够,需保证奖励标签的可靠性。现有数据管道依赖标注者可信,而策略方法假定答案本身正确。本文将扩展视为可验证的数据构建问题,解耦提示难度与答案可靠性两个维度:通过路径特异性进化算子提升提示难度,通过离线假设检验的证伪机制保障答案可信。提出VeriEvol框架,包含类型感知进化模块(生成更难图像引导的提示)和HTV-Agent验证器(需多源反例均失效才接受答案)。该框架可扩展数据量、增加进化路径或验证通道,并直接接入现有GRPO类强化学习流程。在五个基准测试上,将微调数据从10K扩增至250K后,平均准确率由35.42升至54.73;固定主干模型、微调初始化和训练流程后,相比未进化基线,累计提升+3.88,其中+1.82来自进化提示,+2.06来自验证器。所有提示、数据、模型、代码及每条样本的完整验证轨迹均已开源,支持下游扩展与审计。
原文摘要 · Abstract (English)
Scaling reinforcement learning for visual mathematical reasoning requires more than generating harder questions: as data volume grows, the reward labels themselves must remain reliable. Yet existing data pipelines scale supervision while trusting the labeller, and policy-side methods assume the underlying answers are already correct. We instead treat scaling as a verifiable data-construction problem and decouple two axes before any policy update: prompt difficulty, expanded by route-specific evolution operators, and answer reliability, enforced by offline hypothesis-test falsification. We instantiate this as VeriEvol, an iterative framework with two extensible components: a type-aware evolution module that rewrites low-difficulty image-question seeds into harder, image-grounded prompts; and HTV-Agent, a verifier that accepts an answer only after multi-source counter-evidence has failed to refute it. The resulting verified data scales in volume, extends by adding evolution routes or verifier channels, and plugs directly into existing GRPO-style RL recipes. On a five-benchmark visual-math suite, scaling evolved SFT data from 10K to 250K samples raises the mean accuracy from 35.42 to 54.73; then, with backbone, SFT initialization, and GRPO recipe held fixed, VeriEvol adds a cumulative +3.88 over an un-evolved RL baseline, of which +1.82 comes from evolved prompts and +2.06 from the HTV-Agent verifier. We release the prompts, data, models, code, and the full verifier trace of every sample, so that downstream work can scale and audit the pipeline rather than only inspect its outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。