用扩散模型生成更稳定的表格注意力,提升大模型推理准确性。
From Table to Cell: Attention for Better Reasoning with TABALIGN

- 用双向去噪扩散模型生成带掩码的单元格计划,实现显式单元格定位。
- 在8个基准上平均准确率提升15.76个百分点,较强基线显著领先。
- 适合需要高精度表格推理的场景,如智能问答与事实验证。
多步大语言模型在结构化表格上的推理失败,源于规划与执行缺乏显式的单元格对齐机制。现有方法强制规划采用从左到右的分解方式,违背表格排列不变性,且仅根据生成内容评分,忽视单元格定位。我们开展初步研究发现,扩散语言模型(DLMs)在表格上生成的注意力比自回归模型更符合人类认知、更稳定,行重排下注意力-AUROC变异降低40.2%中位数。受此启发,我们提出TABALIGN框架,明确建立规划与执行间的单元格对齐契约。TABALIGN将一个掩码扩散模型规划器(双向去噪生成二值单元格掩码作为计划步骤),与一个轻量级验证器TABATTN结合,后者在1,600条人工验证的注意力标准上训练,通过注意力重叠度评估每一步的合理性。在覆盖表格问答与事实验证的八个基准上,TABALIGN在8B级模型规模下,平均准确率相比最强开源基线提升15.76个百分点;固定推理器的消融实验表明,其中2.87个百分点的增益来自扩散规划器优于自回归规划器;更清晰的规划也使下游推理执行加速44.64%。
原文摘要 · Abstract (English)
Multi-step LLM reasoning over structured tables fails because planning and execution share no explicit cell-grounding contract. Existing methods constrain the planner to a left-to-right factorization at odds with table permutation invariance, and score intermediate states by generated content alone, overlooking cell grounding. We conduct a pilot study showing that diffusion language models (DLMs) produce more human-aligned and permutation-stable cell attention on tables than autoregressive models, with a 40.2% median reduction in attention-AUROC variability under row reordering. Motivated by this, we propose TABALIGN, a planned table reasoning framework that operationalizes the contract. TABALIGN pairs a masked DLM planner, whose bidirectional denoising emits plan steps as binary cell masks, with TABATTN, a lightweight verifier trained on 1,600 human-verified attention standards to score each step by its attention overlap with the plan-designated mask. Across eight benchmarks covering table question answering and fact verification, TABALIGN improves average accuracy by 15.76 percentage points over the strongest open-source baseline at comparable 8B-class scale, with a matched-backbone ablation attributing 2.87 percentage points of this gain to the DLM planner over an AR planner on a fixed reasoner. Cleaner DLM plans also accelerate downstream reasoning execution by 44.64%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。