用可执行代码增强表格理解，让模型推理过程透明可验证。
CoReTab: Improving Multimodal Table Understanding with Code-driven Reasoning
- 将多步推理与可执行Python代码结合，自动生成可验证的标注。
- 在17个任务上分别提升6.2%、5.7%、25.6%准确率。
- 适合需要可解释推理的表格问答与结构理解场景。
现有多模态表格理解数据集（如MMTab）主要提供简短的事实答案，缺乏显式的多步推理监督。在这些数据上训练的模型常生成简略回应，准确率低且推理过程不可解释。我们提出CoReTab——一种代码驱动的推理框架，通过将多步推理与可执行Python代码结合，实现可扩展、可解释且可自动验证的标注。基于该框架，我们构建了包含11.5万条经验证样本的数据集，平均每条回答529个词元，并通过三阶段流水线微调开源多模态大模型。我们在17个MMTab基准任务上评估该模型，涵盖表格问答、事实验证和表格结构理解。结果表明，模型相比原基线分别取得+6.2%、+5.7%、+25.6%的显著提升，同时生成透明可验证的推理路径。这些成果确立了CoReTab作为提升多模态表格理解中多步推理能力的稳健通用监督框架。
原文摘要 · Abstract (English)
Existing datasets for multimodal table understanding, such as MMTab, primarily provide short factual answers without explicit multi-step reasoning supervision. Models trained on these datasets often generate brief responses that offers insufficient accuracy and limited interpretability into how these models arrive at the final answer. We introduce CoReTab, a code-driven reasoning framework that produces scalable, interpretable, and automatically verifiable annotations by coupling multi-step reasoning with executable Python code. Using the CoReTab framework, we curate a dataset of 115K verified samples averaging 529 tokens per response and fine-tune open-source MLLMs through a three-stage pipeline. We evaluate the resulting model trained on CoReTab across 17 MMTab benchmarks spanning table question answering, fact verification, and table structure understanding. Our model achieves significant gains of +6.2%, +5.7%, and +25.6%, respectively, over MMTab-trained baselines, while producing transparent and verifiable reasoning traces. These results establish CoReTab as a robust and generalizable supervision framework for improving multi-step reasoning in multimodal table understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。