让大模型像解题一样一步步推理表格,拒绝胡猜乱蒙。
V-tableR1: Process-Supervised Multimodal Table Reasoning with Critic-Guided Policy Optimization

- 用强化学习引导模型逐步解释表格推理过程
- 在复杂表格任务上超越18倍大的模型,准确率领先
- 适合需要可解释推理的金融、科研等严谨场景
我们提出V-tableR1,一种基于过程监督的强化学习框架,促使多模态大语言模型(MLLMs)进行可验证的严格推理。当前仅基于最终结果训练的MLLM常将视觉推理视为黑箱,依赖表面模式匹配而非多步逻辑推演。尽管可验证奖励的强化学习能约束推理路径,但将其扩展至视觉领域受限于抽象逻辑与连续像素空间之间的模糊对应关系。我们通过利用表格固有的确定性网格结构作为理想视觉测试平台,解决了这一问题。V-tableR1采用专用批判者视觉语言模型(VLM),对策略VLM生成的显式视觉思维链提供密集的步骤级反馈。为优化该系统,提出过程引导直接对齐策略优化(PGPO),融合过程奖励、解耦策略约束和长度感知动态采样。大量实验表明,V-tableR1明确惩罚视觉幻觉和捷径猜测。通过将多模态推理从黑箱模式匹配转变为可验证的逻辑推导,4B规模的V-tableR1在复杂表格基准上达到开源模型中的最先进水平,超越高达18倍大小的模型,并显著优于其SFT基线。
原文摘要 · Abstract (English)
We introduce V-tableR1, a process-supervised reinforcement learning framework that elicits rigorous, verifiable reasoning from multimodal large language models (MLLMs). Current MLLMs trained solely on final outcomes often treat visual reasoning as a black box, relying on superficial pattern matching rather than performing rigorous multi-step inference. While Reinforcement Learning with Verifiable Rewards could enforce transparent reasoning trajectories, extending it to visual domains remains severely hindered by the ambiguity of grounding abstract logic into continuous pixel space. We solve this by leveraging the deterministic grid structure of tables as an ideal visual testbed. V-tableR1 employs a specialized critic VLM to provide dense, step-level feedback on the explicit visual chain-of-thought generated by a policy VLM. To optimize this system, we propose Process-Guided Direct Alignment Policy Optimization (PGPO), a novel RL algorithm integrating process rewards, decoupled policy constraints, and length-aware dynamic sampling. Extensive evaluations demonstrate that V-tableR1 explicitly penalizes visual hallucinations and shortcut guessing. By fundamentally shifting multimodal inference from black-box pattern matching to verifiable logical derivation, V-tableR1 4B establishes state-of-the-art accuracy among open-source models on complex tabular benchmarks, outperforming models up to 18x its size and improving over its SFT baseline
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。