让模型解释为何支持或反驳科学结论,提升可解释性。
Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers
- 将表格与文本对齐转为解释任务,要求定位关键单元格。
- 引入新数据集,标注了支持判断的最小单元格集合。
- 发现大模型虽标签正确,却常无法复现人类推理路径。
科学主张验证通常需要判断主张是否被表格支持。然而,仅预测最终标签不足以揭示模型推理过程,缺乏可解释性。为此,我们把表格-文本对齐重构为解释任务,要求模型识别验证主张所必需的表格单元格。通过扩展SciTab基准,构建新数据集,人工标注了细胞级理由:标注者先判断主张真假,再标出支撑其决策的最小单元格集合。标注完成后,基于收集信息提出处理模糊案例的分类体系。实验表明:(1)引入表格对齐信息能提升主张验证性能;(2)多数大模型虽常预测正确标签,但无法恢复人类一致的理由,表明其预测非源于忠实推理。
原文摘要 · Abstract (English)
Scientific claim verification against tables typically requires predicting whether a claim is supported or refuted given a table. However, we argue that predicting the final label alone is insufficient: it reveals little about the model's reasoning and offers limited interpretability. To address this, we reframe table-text alignment as an explanation task, requiring models to identify the table cells essential for claim verification. We build a new dataset by extending the SciTab benchmark with human-annotated cell-level rationales. Annotators verify the claim label and highlight the minimal set of cells needed to support their decision. After the annotation process, we utilize the collected information and propose a taxonomy for handling ambiguous cases. Our experiments show that (i) incorporating table alignment information improves claim verification performance, and (ii) most LLMs, while often predicting correct labels, fail to recover human-aligned rationales, suggesting that their predictions do not stem from faithful reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。