提升大模型在文档不一致检测中的证据提取能力
Improved Evidence Extraction and Metrics for Document Inconsistency Detection with LLMs
- 提出红笔重试框架与约束过滤，增强证据提取效果
- 新指标显示性能显著优于传统提示方法
- 适合关注大模型可解释性与文档验证的研究者
大语言模型因大规模训练数据和参数量展现出强大能力，但在文档不一致检测方面的研究仍较有限。本文聚焦大模型在该任务中的证据提取能力，引入新的综合性证据提取评估指标，并提出红笔重试框架结合约束过滤机制，显著提升证据提取性能。实验结果表明该方法优于其他提示策略。同时，研究发布了新的半合成数据集，用于评估证据提取效果。
原文摘要 · Abstract (English)
Large language models (LLMs) are becoming useful in many domains due to their impressive abilities that arise from large training datasets and large model sizes. However, research on LLM-based approaches to document inconsistency detection is relatively limited. We address this gap by investigating evidence extraction capabilties of LLMs for document inconsistency detection. To this end, we introduce new comprehensive evidence-extraction metrics and a redact-and-retry framework with constrained filtering that substantially improves evidence extraction performance over other prompting methods. We support our approach with strong experimental results and release a new semi-synthetic dataset for evaluating evidence extraction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。