用强化学习让大模型更准地理解表格,提升推理效率。
Table-R1: Region-based Reinforcement Learning for Table Understanding
- 通过区域增强微调,让模型先定位表格相关区域再回答问题。
- 在多个数据集上平均提升14.36分,小模型性能超过十倍参数的大模型。
- 动态奖励机制减少无效输出,适合需要高效表格推理的场景。
表格因其行列结构交互带来独特挑战,语言模型需专门方法才能有效理解。尽管大语言模型(LLMs)在表格问答中展现潜力,但其性能优化仍不充分。本文提出基于区域的Table-R1,通过区域增强监督微调(RE-SFT)引导模型识别相关表格区域,并融合文本、符号与程序化推理。同时引入表感知组相对策略优化(TARPO),采用混合奖励机制动态平衡区域准确率与答案正确性,结合衰减区域奖励与一致性惩罚以对齐推理过程。实验表明,Table-R1在三个基准数据集上对多个基础模型平均提升14.36分,甚至超越参数量十倍的基线模型;TARPO相比GRPO将响应令牌消耗降低67.5%,显著提升大模型在表格推理中的效率与能力。
原文摘要 · Abstract (English)
Tables present unique challenges for language models due to their structured row-column interactions, necessitating specialized approaches for effective comprehension. While large language models (LLMs) have demonstrated potential in table reasoning through prompting and techniques like chain-of-thought (CoT) and program-of-thought (PoT), optimizing their performance for table question answering remains underexplored. In this paper, we introduce region-based Table-R1, a novel reinforcement learning approach that enhances LLM table understanding by integrating region evidence into reasoning steps. Our method employs Region-Enhanced Supervised Fine-Tuning (RE-SFT) to guide models in identifying relevant table regions before generating answers, incorporating textual, symbolic, and program-based reasoning. Additionally, Table-Aware Group Relative Policy Optimization (TARPO) introduces a mixed reward system to dynamically balance region accuracy and answer correctness, with decaying region rewards and consistency penalties to align reasoning steps. Experiments show that Table-R1 achieves an average performance improvement of 14.36 points across multiple base models on three benchmark datasets, even outperforming baseline models with ten times the parameters, while TARPO reduces response token consumption by 67.5% compared to GRPO, significantly advancing LLM capabilities in efficient tabular reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。