提出可解释的表格问答方法POS,让模型决策过程透明可验证。
Interpretable LLM-based Table Question Answering
- 将问题分解为可执行的SQL步骤,每步结果透明可追踪。
- 在多个基准上准确率媲美顶尖方法,减少25倍LLM调用和查询次数。
- 人类与模型对解释的一致性高达90.59%,适合高风险场景使用。
表格问答(Table QA)的可解释性在金融、医疗等高风险领域至关重要。尽管基于大语言模型(LLMs)的现有方法取得高准确率,但其解释常模糊不清。本文提出计划-SQL(Plan-of-SQLs, POS)方法,将问题分解为一系列原子步骤,每步直接转化为可执行的SQL命令,确保每个中间结果透明可查。实验表明:首先,POS生成的解释质量最高,显著提升用户模拟和验证模型决策的能力;其次,在标准基准(TabFact、WikiTQ、FeTaQA)上,其问答准确率与现有方法相当,同时效率更高——所需LLM调用和数据库查询最多减少25倍,且在大表上表现更鲁棒;最后,基于相同解释,模型与人类在正向模拟中达成高达90.59%的一致性,表明大语言模型可作为人类评估解释的有效代理。
原文摘要 · Abstract (English)
Interpretability in Table Question Answering (Table QA) is critical, especially in high-stakes domains like finance and healthcare. While recent Table QA approaches based on Large Language Models (LLMs) achieve high accuracy, they often produce ambiguous explanations of how answers are derived. We propose Plan-of-SQLs (POS), a new Table QA method that makes the model's decision-making process interpretable. POS decomposes a question into a sequence of atomic steps, each directly translated into an executable SQL command on the table, thereby ensuring that every intermediate result is transparent. Through extensive experiments, we show that: First, POS generates the highest-quality explanations among compared methods, which markedly improves the users' ability to simulate and verify the model's decisions. Second, when evaluated on standard Table QA benchmarks (TabFact, WikiTQ, and FeTaQA), POS achieves QA accuracy that is competitive to existing methods, while also offering greater efficiency-requiring significantly fewer LLM calls and table database queries (up to 25x fewer)-and more robust performance on large-sized tables. Finally, we observe high agreement (up to 90.59% in forward simulation) between LLMs and human users when making decisions based on the same explanations, suggesting that LLMs could serve as an effective proxy for humans in evaluating Table QA explanations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。