让大模型把表格当独立模态，显著提升表格推理能力
Table as a Modality for Large Language Models
- 将表格视为独立模态，用超图神经网络编码结构信息
- 在多个数据集上平均提升42.65%的泛化性能
- 适合需要精确表格理解的任务场景
为将大语言模型（LLMs）的成功迁移到表格推理任务，社区已做出诸多努力。然而，通过在我们提出的 StructQA 基准上的探测实验，我们发现即使最先进的 LLMs（如 GPT 系列）仍难以有效处理表格数据。当前方法通常仅将表格数据与元信息序列化后输入模型，导致结构信息丢失。为此，本文提出 TAMO，主张将表格作为与文本并列的独立模态。TAMO 框架采用超图神经网络作为全局表格编码器，无缝集成主流大语言模型。在 HiTab、WikiTQ、WikiSQL、FeTaQA 以及 StructQA 等多个基准数据集上的实证结果表明，该方法在泛化性上实现显著提升，平均相对增益达 42.65%。
原文摘要 · Abstract (English)
To migrate the remarkable successes of Large Language Models (LLMs), the community has made numerous efforts to generalize them to the table reasoning tasks for the widely deployed tabular data. Despite that, in this work, by showing a probing experiment on our proposed StructQA benchmark, we postulate that even the most advanced LLMs (such as GPTs) may still fall short of coping with tabular data. More specifically, the current scheme often simply relies on serializing the tabular data, together with the meta information, then inputting them through the LLMs. We argue that the loss of structural information is the root of this shortcoming. In this work, we further propose TAMO, which bears an ideology to treat the tables as an independent modality integrated with the text tokens. The resulting model in TAMO is a multimodal framework consisting of a hypergraph neural network as the global table encoder seamlessly integrated with the mainstream LLM. Empirical results on various benchmarking datasets, including HiTab, WikiTQ, WikiSQL, FeTaQA, and StructQA, have demonstrated significant improvements on generalization with an average relative gain of 42.65%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。