提出细粒度视觉等价奖励模型,提升图表代码生成的准确率
Visual-ERM: Reward Modeling for Visual Equivalence
- 构建多模态生成式奖励模型,在渲染空间直接评估视觉质量
- 在图表/表格/SVG解析任务上分别提升8.4、2.7、4.1分
- 适合需要高保真视觉重建的AI系统开发者与研究者
视觉到代码任务需将图表、表格、SVG等结构化视觉输入重建为可执行或结构化表示,保持高视觉保真度。尽管大型视觉语言模型(LVLM)通过监督微调取得良好效果,但强化学习仍受制于奖励信号错位。现有奖励依赖文本规则或粗粒度视觉嵌入相似性,难以捕捉细微视觉差异,且易被奖励劫持。本文提出视觉等价奖励模型(Visual-ERM),一种多模态生成式奖励模型,可在渲染空间直接提供细粒度、可解释、任务无关的反馈,评估视觉到代码质量。集成至强化学习后,使Qwen3-VL-8B-Instruct在图表到代码任务上提升+8.4分,并在表格与SVG解析中平均提升+2.7和+4.1分;进一步通过反思与修正增强测试时扩展能力。我们还引入VisualCritic-RewardBench(VC-RewardBench),用于判断结构化视觉数据上的细粒度图像间差异的基准测试,8B规模的Visual-ERM显著优于Qwen3-VL-235B-Instruct,接近领先闭源模型。结果表明,细粒度视觉奖励监督对视觉到代码的强化学习既是必要也是充分条件。
原文摘要 · Abstract (English)
Vision-to-code tasks require models to reconstruct structured visual inputs, such as charts, tables, and SVGs, into executable or structured representations with high visual fidelity. While recent Large Vision Language Models (LVLMs) achieve strong results via supervised fine-tuning, reinforcement learning remains challenging due to misaligned reward signals. Existing rewards either rely on textual rules or coarse visual embedding similarity, both of which fail to capture fine-grained visual discrepancies and are vulnerable to reward hacking. We propose Visual Equivalence Reward Model (Visual-ERM), a multimodal generative reward model that provides fine-grained, interpretable, and task-agnostic feedback to evaluate vision-to-code quality directly in the rendered visual space. Integrated into RL, Visual-ERM improves Qwen3-VL-8B-Instruct by +8.4 on chart-to-code and yields consistent gains on table and SVG parsing (+2.7, +4.1 on average), and further strengthens test-time scaling via reflection and revision. We also introduce VisualCritic-RewardBench (VC-RewardBench), a benchmark for judging fine-grained image-to-image discrepancies on structured visual data, where Visual-ERM at 8B decisively outperforms Qwen3-VL-235B-Instruct and approaches leading closed-source models. Our results suggest that fine-grained visual reward supervision is both necessary and sufficient for vision-to-code RL, regardless of task specificity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。