用多源矛盾证据提升谣言核查准确率
Contradiction to Consensus: Dual Perspective, Multi Source Retrieval Based Claim Verification with Source Level Disagreement using LLM
- 双视角检索：同时找支持与反证，覆盖更多来源
- 融合维基、PubMed、谷歌数据，提升知识广度
- 可视化源间分歧，增强结果可解释性
虚假信息在数字平台上的传播可能带来重大社会风险。事实核查系统有助于识别潜在误导信息，但其效果受限于所依赖的知识源。多数自动化核查系统仅依赖单一知识源并使用该源的支撑证据，忽视源间分歧，导致知识覆盖不足且透明度低。为此，我们提出一种基于大语言模型（LLMs）、多视角证据检索和跨源分歧分析的开放域事实核查新系统。方法上，引入新颖检索策略，同时获取声明及其否定形式的证据，从维基百科、PubMed 和 Google 汇集支持与反驳信息；经过滤、去重与跨源聚合，构建更全面的知识库。该知识库用于 LLM 进行验证，并通过模型置信度分析量化与可视化源间分歧。在四个基准数据集上，使用五种 LLM 的广泛评估表明，知识聚合不仅提升了核查性能，还揭示了各源推理差异。研究强调在证据构建中融入多样性、矛盾性与聚合的重要性，对构建可靠透明的核查系统具有关键意义。
原文摘要 · Abstract (English)
The spread of misinformation across digital platforms can pose significant societal risks. Claim verification, a.k.a. fact-checking, systems can help identify potential misinformation. However, their efficacy is limited by the knowledge sources that they rely on. Most automated claim verification systems depend on a single knowledge source and utilize the supporting evidence from that source; they ignore the disagreement of their source with others. This limits their knowledge coverage and transparency. To address these limitations, we present a novel system for open-domain claim verification (ODCV) that leverages large language models (LLMs), multi-perspective evidence retrieval, and cross-source disagreement analysis. Our approach introduces a novel retrieval strategy that collects evidence for both the original and the negated forms of a claim, enabling the system to capture supporting and contradicting information from diverse sources: Wikipedia, PubMed, and Google. These evidence sets are filtered, deduplicated, and aggregated across sources to form a unified and enriched knowledge base that better reflects the complexity of real-world information. This aggregated evidence is then used for claim verification using LLMs. We further enhance interpretability by analyzing model confidence scores to quantify and visualize inter-source disagreement. Through extensive evaluation on four benchmark datasets with five LLMs, we show that knowledge aggregation not only improves claim verification but also reveals differences in source-specific reasoning. Our findings underscore the importance of embracing diversity, contradiction, and aggregation in evidence for building reliable and transparent claim verification systems
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。