arXiv:2604.23483cs.AI2026-04

提出黑箱攻击框架,用语义改写骗过主流NLP系统

Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelines

论文配图:Agentic Adversarial Rewriting Exposes Architectural Vulnerabilities in Black-Box NLP Pipelines
图 1 · 摘自论文原文
  • 双代理协作在语义空间生成保意改写,仅靠二元反馈和10次查询
  • 对大模型系统成功绕过率19.95%~40.34%,远超传统方法的3.90%
  • 揭示架构缺陷与攻击成功率关系,适合安全评估与防御研究

多组件自然语言处理(NLP)流水线广泛应用于高风险决策场景,但现有对抗方法无法在真实条件下测试其鲁棒性:仅能获得二元反馈、无梯度访问且查询预算严格受限。本文正式定义此严苛的黑箱威胁模型,提出一个双代理逃逸框架,在语义扰动空间中运行。攻击者代理生成语义保持的重写文本,提示优化代理则基于仅有的二元决策反馈,在10次查询内优化攻击策略。在四个基于证据的虚假信息检测流水线上的评估显示,该框架对现代大语言模型(LLM)系统实现了19.95%至40.34%的逃逸率,而依赖代理模型的词级扰动基线最多仅达3.90%。依赖静态词汇检索的旧系统几乎完全脆弱,逃逸率达97.02%,确立了下限。逃逸效果与三个架构特性相关:证据检索机制、检索-推理耦合方式及基线分类准确率。迭代提示优化对最鲁棒目标带来最大边际提升,证实自适应策略发现对复杂逃逸至关重要。对成功改写的分析揭示四种利用模式,分别针对流水线不同阶段的漏洞。基于模式的防御可将逃逸率降低最高65.18%。

原文摘要 · Abstract (English)

Multi-component natural language processing (NLP) pipelines are increasingly deployed for high-stakes decisions, yet no existing adversarial method can test their robustness under realistic conditions: binary-only feedback, no gradient access, and strict query budgets. We formalize this strict black-box threat model and propose a two-agent evasion framework operating in a semantic perturbation space. An Attacker Agent generates meaning-preserving rewrites while a Prompt Optimization Agent refines the attack strategy using only binary decision feedback within a 10-query budget. Evaluated against four evidence-based misinformation detection pipelines, the framework achieves evasion rates of 19.95 to 40.34% on modern large language model (LLM) based systems, compared to at most 3.90% for token-level perturbation baselines that rely on surrogate models because they cannot operate under our threat model. A legacy system relying on static lexical retrieval exhibits near-total vulnerability 97.02%, establishing a lower bound that exposes how architectural choices govern the attack surface. Evasion effectiveness is associated with three architectural properties: evidence retrieval mechanism, retrieval-inference coupling, and baseline classification accuracy. The iterative prompt optimization yields the largest marginal gains against the most robust targets, confirming that adaptive strategy discovery is essential when evasion is non-trivial. Analysis of successful rewrites reveals four exploitation patterns, each targeting failures at distinct pipeline stages. A pattern-informed defense reduces the evasion rate by up to 65.18%.

黑箱攻击大模型安全语义改写防御评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。