让大模型理解表格层级结构,提升问答准确率。
DeepTable: Structural Attention Biases and Tree Path Encoding for Hierarchical Table Understanding

- 用可学习的注意力偏置显式建模行/列共享关系。
- 通过树路径编码保留多级表头位置信息,平均提分7.42点。
- 适合需要精准理解复杂表格的AI研究者使用。
大型语言模型在表格理解任务中表现优异,但通常将表格内容和表头线性化为词元序列,弱化了二维及层级结构关系。现有参数高效微调方法虽引入基础行列信息,但未显式捕捉由多级表头引发的丰富结构依赖。本文提出DeepTable,一种面向大模型的结构感知表格理解方法,包含两个互补组件:结构注意力偏置(SAB)通过在注意力得分中引入可学习偏置,显式表示表元对是否共享同一行或列;树路径编码(TPE)则用行、列表头的祖先路径表示每个表元,保留其在多级结构中的位置。我们将DeepTable与TableLoRA(He et al., 2025)结合,在三种大模型主干上注入结构信息。在三个表格问答基准测试中,DeepTable持续优于对应TableLoRA基线,分别在HiTab上平均提升7.42分,在WikiTQ上提升3.23分,在FeTaQA上提升2.01个BLEU点。结果证明所提结构偏置在不同大模型上的有效性。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated strong performance in table understanding. However, they typically process table content and headers as linearized token sequences. This representation weakens the two-dimensional and hierarchical structural relationships encoded by multi-level row and column headers. Existing parameter-efficient fine-tuning methods incorporate basic row and column information but do not explicitly capture the rich structural dependencies induced by hierarchical table headers. We propose DeepTable, a structure-aware approach for table understanding with LLMs. DeepTable comprises two complementary components. Structural Attention Bias (SAB) introduces learnable biases into the attention logits to explicitly represent whether pairs of table tokens share the same row or column. Tree Path Encoding (TPE) represents each table token using the ancestor paths of its row and column headers, preserving its position within the multi-level table structure. We integrate DeepTable with TableLoRA (He et al., 2025) to inject structural information into parameter-efficient adaptation. Across three LLM backbones, DeepTable consistently improves the corresponding TableLoRA baselines on three table question answering benchmarks, achieving average gains of 7.42 points on HiTab, 3.23 points on WikiTQ, and 2.01 BLEU points on FeTaQA. These results demonstrate the effectiveness of the proposed structural biases across different LLM backbones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。