对比表格图文输入对问答效果的影响，发现最佳方案随问题复杂度和表大小变化。
Texts or Images? A Fine-grained Analysis on the Effectiveness of Input Representations and Models for Table Question Answering
- 控制变量对比图文输入在不同模型下的表现
- 动态选择输入形式可提升平均性能10%
- 适合研究表格问答输入策略的学者和开发者
在表格问答（TQA）任务中，表格可作为文本或图像输入。以往研究表明，将表格图像输入多模态大语言模型（MLLMs）的表现与文本输入大语言模型（LLMs）相当甚至更优。然而，缺乏受控实验设置限制了对两种方法的精细区分。本文首次在两个维度（问题复杂度和表大小）上开展受控研究，分析多种MLLM与LLM组合的效果。我们基于现有TQA数据集构建新基准，在七组模型对比中发现最优输入形式随场景变化。提出FRES方法，根据情况动态选择输入形式，相比盲目使用任一形式，平均性能提升10%。
原文摘要 · Abstract (English)
In table question answering (TQA), tables are encoded as either texts or images. Prior work suggests that passing images of tables to multi-modal large language models (MLLMs) performs comparably to or even better than using textual input with large language models (LLMs). However, the lack of controlled setups limits fine-grained distinctions between these approaches. In this paper, we conduct the first controlled study on the effectiveness of several combinations of table representations and models from two perspectives: question complexity and table size. We build a new benchmark based on existing TQA datasets. In a systematic analysis of seven pairs of MLLMs and LLMs, we find that the best combination of table representation and model varies across setups. We propose FRES, a method selecting table representations dynamically, and observe a 10% average performance improvement compared to using both representations indiscriminately.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。