用检索增强大模型检测长篇新冠谣言，准确率显著提升。
Use of Retrieval-Augmented Large Language Model Agent for Long-Form COVID-19 Fact-Checking
- 构建双代理系统，先提取主张再用13万篇文献检索验证。
- 在50篇假新闻上表现超越基线模型，一致性达0.629。
- 适合公共卫生、信息审核等需要高可信度的场景。
新冠疫情信息泛滥亟需可扩展的准确事实核查方案。本文提出SAFE系统，结合大语言模型与检索增强生成（RAG），提升长篇新冠虚假信息的自动化核查能力。SAFE包含两个代理：一个用于主张提取，另一个通过基于13万篇新冠研究文献的LOTR-RAG进行主张验证。改进版SAFE（LOTR-RAG + SRAG）引入自检索（Self-RAG）优化查询重写。在50篇假新闻（2-17页）上评估，共含246个标注主张（均值4.922，标准差3.186），由公共卫生专家标注为真（14.1%）、部分真（14.4%）、假（27.0%）、部分假（2.2%）、误导性（21.0%）。所有指标上，SAFE均显著优于基线大模型（p < 0.001）。一致性得分（0-1）中，SAFE（LOTR-RAG）达0.629，高于SAFE（+SRAG）的0.577和基线的0.279。主观评分（0-4级）显示，其在实用性（3.640）、清晰度（3.800）、真实性（3.526）上最高。引入SRAG仅小幅提升清晰度，整体性能略有下降。结果表明，该系统有效克服了大模型在一致性和可解释性上的局限，核心LOTR-RAG设计优于增强版本，为可扩展的信息误传应对提供了坚实基础。
原文摘要 · Abstract (English)
The COVID-19 infodemic calls for scalable fact-checking solutions that handle long-form misinformation with accuracy and reliability. This study presents SAFE (system for accurate fact extraction and evaluation), an agent system that combines large language models with retrieval-augmented generation (RAG) to improve automated fact-checking of long-form COVID-19 misinformation. SAFE includes two agents - one for claim extraction and another for claim verification using LOTR-RAG, which leverages a 130,000-document COVID-19 research corpus. An enhanced variant, SAFE (LOTR-RAG + SRAG), incorporates Self-RAG to refine retrieval via query rewriting. We evaluated both systems on 50 fake news articles (2-17 pages) containing 246 annotated claims (M = 4.922, SD = 3.186), labeled as true (14.1%), partly true (14.4%), false (27.0%), partly false (2.2%), and misleading (21.0%) by public health professionals. SAFE systems significantly outperformed baseline LLMs in all metrics (p < 0.001). For consistency (0-1 scale), SAFE (LOTR-RAG) scored 0.629, exceeding both SAFE (+SRAG) (0.577) and the baseline (0.279). In subjective evaluations (0-4 Likert scale), SAFE (LOTR-RAG) also achieved the highest average ratings in usefulness (3.640), clearness (3.800), and authenticity (3.526). Adding SRAG slightly reduced overall performance, except for a minor gain in clearness. SAFE demonstrates robust improvements in long-form COVID-19 fact-checking by addressing LLM limitations in consistency and explainability. The core LOTR-RAG design proved more effective than its SRAG-augmented variant, offering a strong foundation for scalable misinformation mitigation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。