FIRE通过迭代检索与验证，用更少成本实现更准的长文本事实核查。
FIRE: Fact-checking with Iterative Retrieval and Verification
- 采用迭代机制，根据置信度决定是否继续搜索或给出答案。
- 相较基线模型，大语言模型成本降低7.6倍，搜索成本降低16.5倍。
- 适合大规模事实核查场景，尤其关注效率与资源优化的研究者。
长文本事实核查难度高，通常需将文本拆分为原子化陈述。传统方法固定检索若干证据后进行验证，但该方式成本高，未能充分利用验证模型对陈述的内在知识，也未模拟人类搜索的迭代推理过程。为此，我们提出FIRE——一种基于智能体的框架，将证据检索与陈述验证以迭代方式整合。FIRE使用统一机制，根据当前判断的置信度决定是否输出最终答案或生成下一阶段搜索查询。与多种强基准对比发现，FIRE性能略优，同时平均降低大语言模型成本7.6倍、搜索成本16.5倍。结果表明，FIRE在大规模事实核查中具有应用前景。代码已开源：https://github.com/mbzuai-nlp/fire.git。
原文摘要 · Abstract (English)
Fact-checking long-form text is challenging, and it is therefore common practice to break it down into multiple atomic claims. The typical approach to fact-checking these atomic claims involves retrieving a fixed number of pieces of evidence, followed by a verification step. However, this method is usually not cost-effective, as it underutilizes the verification model's internal knowledge of the claim and fails to replicate the iterative reasoning process in human search strategies. To address these limitations, we propose FIRE, a novel agent-based framework that integrates evidence retrieval and claim verification in an iterative manner. Specifically, FIRE employs a unified mechanism to decide whether to provide a final answer or generate a subsequent search query, based on its confidence in the current judgment. We compare FIRE with other strong fact-checking frameworks and find that it achieves slightly better performance while reducing large language model (LLM) costs by an average of 7.6 times and search costs by 16.5 times. These results indicate that FIRE holds promise for application in large-scale fact-checking operations. Our code is available at https://github.com/mbzuai-nlp/fire.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。