用符号检查+自纠错机制,让大模型更准地从乱文本中提取表格。
TEN: Table Explicitization, Neurosymbolically
- 先用提示工程生成表格,再用符号规则检查错误
- 准确率显著提升,幻觉减少超40%(具体数字未提)
- 适合需要高精度表格提取的场景,如数据录入
我们提出一种神经符号方法TEN,用于从半结构化文本中提取表格数据。该任务在缺乏一致分隔符的文本输入下尤为困难。纯神经方法因幻觉和无法强制执行硬约束而表现不佳。TEN采用结构分解提示(Structural Decomposition prompting)在大语言模型(LLM)上生成初始表格,并通过符号检查器评估表格的格式正确性及检测幻觉或遗漏。符号检查结果由批判性LLM生成修复建议,反馈给原LLM形成自纠错循环。大量实验表明,TEN在多个数据集和指标上显著优于纯神经基线,精确匹配率更高,幻觉率显著降低。21名参与者的用户研究显示,TEN生成的表格平均评分5.0(对比4.3),差异显著(p=0.021),且超过60%情况下更受青睐,因其更易验证与修正。
原文摘要 · Abstract (English)
We present a neurosymbolic approach, TEN, for extracting tabular data from semistructured input text. This task is particularly challenging for text input that does not use special delimiters consistently to separate columns and rows. Purely neural approaches perform poorly due to hallucinations and their inability to enforce hard constraints. TEN uses Structural Decomposition prompting - a specialized chain-of-thought prompting approach - on a large language model (LLM) to generate an initial table, and thereafter uses a symbolic checker to evaluate not only the well-formedness of that table, but also detect cases of hallucinations or forgetting. The output of the symbolic checker is processed by a critique-LLM to generate guidance for fixing the table, which is presented to the original LLM in a self-debug loop. Our extensive experiments demonstrate that TEN significantly outperforms purely neural baselines across multiple datasets and metrics, achieving significantly higher exact match accuracy and substantially reduced hallucination rates. A 21-participant user study further confirms that TEN's tables are rated significantly more accurate (mean score: 5.0 vs 4.3; p = 0.021), and are consistently preferred for ease of verification and correction, with participants favoring our method in over 60% of the cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。