用大模型生成骗过谣言检测系统的改写句子,不改变原意却让系统误判。
CAMOUFLAGE: Exploiting Misinformation Detection Systems Through LLM-driven Adversarial Claim Transformation
- 设计双智能体系统,通过迭代改写句子来干扰证据检索和比对模块。
- 在四个真实系统上平均攻击成功率46.92%,保持语义一致。
- 仅依赖模型二分类结果优化,无需梯度或大量查询,适合实际攻击评估。
自动化基于证据的谣言检测系统在评估短声明真实性时,缺乏对其对抗性漏洞的全面分析。现有基于文本的黑盒攻击主要依赖梯度或逻辑值优化的词级替换,无法有效攻破这类包含检索与对比双重模块的系统。本文提出CAMOUFLAGE,一种基于大模型的迭代式攻击方法,采用提示优化代理与攻击者代理双机制:攻击者生成语义等价但可误导检测器的改写句,提示优化代理分析失败案例并改进攻击提示。该方法实现更大结构与风格的文本变换,而非简单替换。不同于以往方法,它仅依据模型的二分类输出进行优化,无需分类器输出逻辑值或频繁调用。在两个学术系统和两个真实世界API上测试,平均攻击成功率达46.92%,同时保持原文语义连贯性与等价性。
原文摘要 · Abstract (English)
Automated evidence-based misinformation detection systems, which evaluate the veracity of short claims against evidence, lack comprehensive analysis of their adversarial vulnerabilities. Existing black-box text-based adversarial attacks are ill-suited for evidence-based misinformation detection systems, as these attacks primarily focus on token-level substitutions involving gradient or logit-based optimization strategies, which are incapable of fooling the multi-component nature of these detection systems. These systems incorporate both retrieval and claim-evidence comparison modules, which requires attacks to break the retrieval of evidence and/or the comparison module so that it draws incorrect inferences. We present CAMOUFLAGE, an iterative, LLM-driven approach that employs a two-agent system, a Prompt Optimization Agent and an Attacker Agent, to create adversarial claim rewritings that manipulate evidence retrieval and mislead claim-evidence comparison, effectively bypassing the system without altering the meaning of the claim. The Attacker Agent produces semantically equivalent rewrites that attempt to mislead detectors, while the Prompt Optimization Agent analyzes failed attack attempts and refines the prompt of the Attacker to guide subsequent rewrites. This enables larger structural and stylistic transformations of the text rather than token-level substitutions, adapting the magnitude of changes based on previous outcomes. Unlike existing approaches, CAMOUFLAGE optimizes its attack solely based on binary model decisions to guide its rewriting process, eliminating the need for classifier logits or extensive querying. We evaluate CAMOUFLAGE on four systems, including two recent academic systems and two real-world APIs, with an average attack success rate of 46.92\% while preserving textual coherence and semantic equivalence to the original claims.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。