用大模型生成相反理由,让假新闻验证更准更透明。
CRAVE: A Conflicting Reasoning Approach for Explainable Claim Verification Using LLMs
- 让大模型从四个维度分析证据,生成对立推理
- 在两个数据集上准确率超越现有方法,且解释性强
- 适合需要可解释性的虚假信息检测场景
虚假信息在数字媒体和人工智能生成内容推动下快速传播,自动化事实核查变得至关重要。传统方法依赖人工标注证据,成本高且难以扩展。尽管近期自动系统有所改进,仍难处理需细致推理的复杂声明。为此,我们提出CRAVE——一种基于大语言模型(LLMs)生成冲突推理的可解释声明验证方法。CRAVE采用三模块框架:首先,通过增强的模糊消除与实体搜索,从维基百科等外部源获取相关证据;其次,利用LLMs在直接证据、语义关系、语言模式和逻辑推理四个维度上生成对立立场的推理并做出初步判断;最后,微调的小语言模型(SLM)根据初步判断评估不同推理的置信度,输出最终真实性判定。该方法能捕捉复杂声明中的细微矛盾,提升验证准确率与透明度。在两个公开数据集上的实验表明,CRAVE性能显著优于现有最优方法,在证据检索和预测解释方面表现更优。代码已开源:https://github.com/8zym/CRAVE。
原文摘要 · Abstract (English)
The rapid spread of misinformation, driven by digital media and AI-generated content, has made automatic claim verification essential. Traditional methods, which depend on expert-annotated evidence, are labor-intensive and not scalable. Although recent automated systems have improved, they still struggle with complex claims that require nuanced reasoning. To address this, we propose CRAVE, a Conflicting Reasoning Approach for explainable claim VErification, that verify the complex claims based on the conflicting rationales reasoned by large language models (LLMs). Specifically, CRAVE introduces a three-module framework. Ambiguity Elimination enchanced Evidence Retrieval module performs ambiguity elimination and entity-based search to gather relevant evidence related to claim verification from external sources like Wikipedia. Conflicting Perspective Reasoning and Preliminary Judgment module with LLMs adopts LLMs to reason rationales with conflicting stances about claim verification from retrieved evidence across four dimensions, i.e., direct evidence, semantic relationships, linguistic patterns, and logical reasoning and make a preliminary judgment. Finally, Small Language Model (SLM) based Judge module is fine-tuned to make use of preliminary judgment from LLMs to assess the confidence of the conflicting rationales and make a final authenticity judgment. This methodology allows CRAVE to capture subtle inconsistencies in complex claims, improving both the accuracy and transparency of claim verification. Extensive experiments on two public claim verification datasets demonstrate that our CRAVE model achieves much better performance than state-of-the-art methods and exhibits a superior capacity for finding relevant evidence and explaining the model predictions. The code is provided at https://github.com/8zym/CRAVE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。