提出轻量评估框架GLEAN，让小模型表格推理更抗污染、可诊断。
GLEAN: Grounded Lightweight Evaluation Anchors for Contamination-Aware Tabular Reasoning
- 用可执行的SQL锚点做评估基准，确保结果可复现。
- 发现TAPEX易出定位错误，TAPAS更倾向胡编或跳过回答。
- 适合关注小模型评估可靠性与错误归因的研究者。
表格推理评测常混杂语义推理、数值计算和脆弱的格式问题，而小模型评估仍易受污染、数据集伪影和检索失败影响。本文提出GLEAN，一种轻量级评估协议，整合污染感知探针、弱监督治理、检索-推理诊断及结构化错误归因，在16GB GPU预算下评估TabFact、WTQ（通过Squall）、TableBench、RobuT和SciTab。以Squall的黄金SQL为可执行锚点（95.2%可执行），GLEAN构建确定性错误分类体系（L0-L4及L0.5上下文缺失），揭示稳定错误模式分离：TAPEX错误偏于定位（L3），TAPAS则偏向幻觉/回避（L2/L0）。通过证据行启发式验证与SQL推导行对比，简单查询精度0.62、召回0.71，混合召回达0.81；且检索召回@K可饱和而端到端准确率仍低，说明需超越原始召回进行归因。我们发布模块化框架，含审计与敏感性检测，提升小模型表格评估的抗污染性和可诊断性。
原文摘要 · Abstract (English)
Tabular reasoning benchmarks mix semantic inference, numerical computation, and brittle table formatting, yet evaluations for small models remain vulnerable to contamination, dataset artifacts, and retrieval failures. We propose GLEAN, a lightweight evaluation protocol that integrates contamination-aware probes, weak-supervision governance, retrieval-reasoning diagnostics, and structured error attribution under tight hardware constraints. We evaluate across TabFact, WTQ via Squall, TableBench, RobuT, and SciTab under a 16GB GPU budget. Using Squall gold SQL as an executable anchor (95.2% execution), GLEAN assigns a deterministic error taxonomy (L0-L4 plus L0.5 context miss) and reveals a stable error-mode separation: TAPEX errors skew toward grounding (L3) while TAPAS errors skew toward hallucination/abstention (L2/L0). We validate evidence-row heuristics against SQL-derived rows on simple queries (0.62 precision / 0.71 recall; hybrid recall 0.81) and show that retrieval Recall@K can saturate even when end-to-end EM/F1 remains limited, motivating attribution beyond raw recall. We release a modular framework with audits and sensitivity checks to make small-model tabular evaluation more contamination-aware and diagnostic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。