arXiv:2607.10402cs.CRcs.AI2026-07

LLM让假信息从内容造假升级为系统性威胁,本文构建框架统一分析攻防漏洞。

Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability

论文配图:Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability
图 1 · 摘自论文原文
  • 提出角色-层级框架,分类LLM在假信息生态中的攻击、防御与脆弱角色
  • 揭示LLM对证据源和验证流程的破坏力,现有检测方法存在可被操纵风险
  • 适合关注虚假信息治理、AI安全及可信验证系统的研究人员与从业者

大型语言模型(LLMs)已将假信息问题从单纯的内容伪造,演变为更广泛的生态系统级安全挑战。当被滥用时,LLMs不仅生成虚假内容,还威胁社会语境、证据来源、检索语料库和验证工作流等假信息防御所依赖的环节。本文提出一个角色-层级框架,统一刻画此类风险与应对策略:角色维度将LLM分为攻击者、防御者与验证系统的脆弱组件;层级维度涵盖内容、社会语境、证据环境与验证流程。基于该框架,我们梳理了基于LLM的攻击手段,研究了基于LLM的检测与验证方法,分析了以LLM为核心的检测范式中的漏洞,并讨论了现有对抗措施。在此基础上,我们识别出三大关键开放挑战:从静态检测准确率转向预算约束下的生态系统级风险评估,强化以LLM为核心的验证流程免受对抗操控,以及部署可审计的人机协同验证系统以实现可信的现实世界假信息防御。

原文摘要 · Abstract (English)

Large language models (LLMs) have transformed misinformation from a primarily content-centric problem into a broader ecosystem-level security challenge. When misused, LLMs create risks beyond false content generation, enabling attacks on the social contexts, evidence sources, retrieval corpora, and verification workflows that misinformation defense depends on. In this paper, we introduce a role-layer framework to unify these risks and defenses. The role dimension characterizes LLMs as attackers, defenders, and vulnerable components of verification systems, while the layer dimension covers content, social contexts, evidence environments, and verification workflows. Guided by this framework, we organize LLM-enabled attacks, investigate LLM-based detection and verification methods, analyze vulnerabilities in LLM-centric detection paradigms, and discuss existing countermeasures against LLM-enabled attacks. Building on this synthesis, we identify three key open challenges: moving from static detection accuracy to budgeted ecosystem-level risk evaluation, hardening LLM-centered verification pipelines against adversarial manipulation, and deploying auditable human-in-the-loop verification systems for trustworthy real-world misinformation defense.

假信息治理LLM安全验证系统人工智能伦理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。