用可信上下文提升大模型信息抽取的准确性，减少幻觉。
SafePassage: High-Fidelity Information Extraction with Black Box LLMs
- 通过生成与原文一致的可信上下文，验证抽取信息的真实性。
- 在任务中将幻觉率降低85%，且误报率极低。
- 仅需1-2小时标注即可训练高效判别模型，适合实际部署。
黑箱大语言模型（LLM）虽使信息抽取配置简便，但难以保证提取内容的真实性。与传统抽取流程不同，其输出未必基于原文。为此，本文提出“安全段落”概念：由模型生成、既与原文一致又与抽取结果相符的上下文。该思想通过三步流程实现——（1）使用LLM提取结构化实体及其上下文；（2）字符串级全局对齐模块；（3）评分模型。实验表明，三者协同可使信息抽取任务中的幻觉率下降高达85%，同时几乎不误判非幻觉内容。安全段落管道与人工判断高度一致，可双重用于评估LLM性能。意外发现：仅用少量任务样本微调的Transformer编码器，在识别不安全段落上表现优于大型语言模型评分器。相关标注可在1-2小时内完成。
原文摘要 · Abstract (English)
Black box large language models (LLMs) make information extraction (IE) easy to configure, but hard to trust. Unlike traditional information extraction pipelines, the information "extracted" is not guaranteed to be grounded in the document. To prevent this, this paper introduces the notion of a "safe passage": context generated by the LLM that is both grounded in the document and consistent with the extracted information. This is operationalized via a three-step pipeline, SafePassage, which consists of: (1) an LLM extractor that generates structured entities and their contexts from a document, (2) a string-based global aligner, and (3) a scoring model. Results show that using these three parts in conjunction reduces hallucinations by up to 85% on information extraction tasks with minimal risk of flagging non-hallucinations. High agreement between the SafePassage pipeline and human judgments of extraction quality mean that the pipeline can be dually used to evaluate LLMs. Surprisingly, results also show that using a transformer encoder fine-tuned on a small number of task-specific examples can outperform an LLM scoring model at flagging unsafe passages. These annotations can be collected in as little as 1-2 hours.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。