用分阶段多模态合成提升表格推理能力,兼顾视觉感知与符号计算。
CFMS: A Coarse-to-Fine Multimodal Synthesis Framework for Enhanced Tabular Reasoning
- 先粗后细:先用多模态大模型生成多视角知识包,再用符号引擎精准操作表格
- 在WikiTQ和TabFact上表现优异,尤其在大表和小模型下仍稳定有效
- 适合需要高鲁棒性表格推理的场景,如问答系统、事实验证
表格推理是问答和事实验证等任务的关键能力,需理解自由文本问题与半结构化表格。现有方法如链式思维(CoT)虽引入推理链,但纯符号方法难以捕捉整体视觉模式。为此,我们提出粗到精多模态合成框架(CFMS),采用两阶段范式,将高层视觉感知与细粒度符号推理分层解耦。在粗粒度阶段,CFMS利用多模态大语言模型(MLLMs)一次性生成多视角知识元组,该元组作为动态推理地图引导细粒度阶段;在此阶段,符号引擎对表格执行目标明确且高效的迭代操作。在WikiTQ与TabFact基准上的大量实验表明,CFMS达到有竞争力的准确率。该框架在处理大型表格及使用较小主干模型时表现出显著鲁棒性,验证了其有效性与泛化能力。
原文摘要 · Abstract (English)
Reasoning over tabular data is a crucial capability for tasks like question answering and fact verification, as it requires models to comprehend both free-form questions and semi-structured tables. However, while methods like Chain-of-Thought (CoT) introduce reasoning chains, purely symbolic methodes are inherently limited by their blindness to holistic visual patterns. To address this, we propose the Coarse-to-Fine Multimodal Synthesis framework (CFMS), a novel two-stage paradigm that hierarchically decouples high-level visual perception from granular symbolic reasoning. In the Coarse Stage, CFMS leverages the Multimodal Large Language Models (MLLMs) to perform a one-time synthesis of a multi-perspective knowledge tuple. This tuple subsequently serves as a dynamic reasoning map to guide the fine stage, where a symbolic engine executes a targeted and efficient sequence of iterative operations over the table. Extensive experiments on the WikiTQ and TabFact benchmarks demonstrate that CFMS achieves competitive accuracy. The framework exhibits particular robustness when handling large tables and when instantiated with smaller backbone models, validating its effectiveness and generalizability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。