发现大模型读表常出错，提出用小模型纠错提升准确率
When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors
- 用专门批评模型检测大模型读表时的数据引用错误
- 纠错后答案准确率最高提升12.0%，小模型检测F1达78.2%
- 适合关注大模型推理可靠性与可解释性的研究者
尽管大语言模型在表格任务上表现良好，但仍存在数据引用错误（DREs），即错误引用或遗漏表格数值，尽管它们理解表格结构。这类错误直接影响中间推理步骤的正确性与可靠性。此前研究仅进行了有限的小规模分析。本文首次系统评估了不同模型和任务下的表格数据引用错误。结果表明，所有测试模型（参数量1.7B至20B）均存在此类错误。我们进一步证明，引入数据引用作为评判标准可显著提升答案准确率，最高达12.0%，通过批评模型过滤与拒绝采样实现。最后，我们训练了一个4B参数的轻量级批评模型，在检测分布内与分布外的DREs时平均F1达到78.2%，有效辅助大模型推理。
原文摘要 · Abstract (English)
While large language models (LLMs) perform well on table tasks, they still make data referencing errors (DREs), i.e., incorrectly citing or omitting table values, despite understanding the table structure. Beyond final-answer accuracy, DREs directly compromise the correctness and reliability of intermediate reasoning steps. Yet prior studies have only offered limited, small-scale analyses. In this work, we present the first systematic evaluation of tabular data referencing errors across different models and tasks. Our results show that DREs occur across all tested models (1.7B to 20B parameters). Furthermore, we demonstrate that incorporating data referencing as a critic significantly improves answer accuracy up to 12.0%, through critic-based filtering and rejection sampling. Finally, we trained a lightweight 4B-parameter critic model that achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-distribution DREs, and effectively assists inference for larger models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。