用多个AI代理协作识别假信息,比单个大模型更准。
Multi-Agentic System Leveraging Open-Source LLMs to Mitigate Disinformation Threats
- 模拟人类标注员分工协作,通过共识与认知多样性提升判断力。
- 在英、波、斯、保等多语言数据集上表现优于GPT-3.5和GPT-4。
- 使用开源模型,透明可查,适合关注信息真实性与安全的团队。
当代社会中,虚假信息威胁已达到令人警觉的程度,由电子通信、社交媒体及人工智能技术的发展所加剧。因此亟需有效对策应对。然而问题规模庞大,人工核查已无法胜任,亟需自动化检测手段。本文提出一种基于多智能体系统的新型方法,模拟人类标注员在识别虚假信息时的决策过程。系统融合共识机制、认知多样性与知识多样性,并借鉴人类标注的层级结构,在包括英语(高资源)、波兰语(中资源)、斯洛伐克语(低资源)和保加利亚语(低资源)在内的多语言数据集上进行评估。实验涵盖直接虚假信息检测、值得验证文本识别以及可验证事实陈述检测任务。结果表明,该系统性能优于单一大型语言模型(如GPT-4与GPT-3.5),且依托开源模型(如LLaMA、Kimi、Qwen、Deepseek、LLaMA-Nemotron)实现更高透明度。
原文摘要 · Abstract (English)
In contemporary societies, the threat of disinformation has reached alarming levels, exacerbated by the proliferation of electronic communication, social media, and advancements in artificial intelligence. As a result, there is an urgent need to develop effective countermeasures to mitigate this menace. However, the sheer scale of the problem renders manual fact-checking and human-based verification inadequate, underscoring the necessity for automated methods to detect and debunk disinformation. This article proposes a novel approach based on a multi-agent system that emulates the decision-making processes of human annotators engaged in disinformation detection tasks. By incorporating a consensus mechanism, diversity in cognition and diversity in knowledge, and also hierarchical structure, inspired by human annotators' behavior, the proposed method achieves superior results compared to individual Large Language Models (LLMs), including GPT 4 and GPT 3.5. The system leverages open models (e.g., LLaMA, Kimi, Qwen, Deepseek and LLaMA-Nemotron) to ensure greater transparency. The evaluation of the proposed method encompasses datasets in languages with varying resource availability, including English (high-resource), Polish (medium-resource), Slovak (low-resource) and Bulgarian (low-resource). Experiments were conducted on tasks such as direct disinformation detection, identification of texts worthy of verification, and detection of texts containing verifiable factual claims.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。