拆解提示词结构,发现不同部分抗攻击能力差异大。
Are All Prompt Components Value-Neutral? Understanding the Heterogeneous Adversarial Robustness of Dissected Prompt in Large Language Models
- 将提示词分解为功能组件,针对性地进行扰动攻击。
- 在5个主流大模型上实现最高攻击成功率,超越现有方法。
- 适合研究模型安全与提示工程的开发者使用。
基于提示词的对抗攻击已成为评估大语言模型鲁棒性的有效手段。然而,现有方法常将提示词视为整体文本,忽视其内部结构异质性——不同组件对对抗鲁棒性的贡献不均。先前工作如PromptRobust假设提示词价值中立,但我们的分析表明,结构复杂、领域特定的提示词中各组件存在差异化的脆弱性。为此,我们提出PromptAnatomy框架,可自动将提示词拆解为功能组件,并通过提出的ComPerturb方法选择性扰动各组件,生成多样且可解释的对抗样本。为保证语言合理性并缓解分布偏移,进一步引入基于困惑度(PPL)的过滤机制。作为补充资源,我们使用PromptAnatomy标注了四个公开的指令微调数据集,并经人工审核验证。在这些数据集及五种先进LLM上的实验表明,ComPerturb实现了当前最优的攻击成功率。消融实验证明了提示词拆解与PPL过滤的互补优势。结果强调了提示词结构感知与可控扰动对可靠对抗鲁棒性评估的重要性。代码与数据已开源:https://github.com/Yujiaaaaa/PACP。
原文摘要 · Abstract (English)
Prompt-based adversarial attacks have become an effective means to assess the robustness of large language models (LLMs). However, existing approaches often treat prompts as monolithic text, overlooking their structural heterogeneity-different prompt components contribute unequally to adversarial robustness. Prior works like PromptRobust assume prompts are value-neutral, but our analysis reveals that complex, domain-specific prompts with rich structures have components with differing vulnerabilities. To address this gap, we introduce PromptAnatomy, an automated framework that dissects prompts into functional components and generates diverse, interpretable adversarial examples by selectively perturbing each component using our proposed method, ComPerturb. To ensure linguistic plausibility and mitigate distribution shifts, we further incorporate a perplexity (PPL)-based filtering mechanism. As a complementary resource, we annotate four public instruction-tuning datasets using the PromptAnatomy framework, verified through human review. Extensive experiments across these datasets and five advanced LLMs demonstrate that ComPerturb achieves state-of-the-art attack success rates. Ablation studies validate the complementary benefits of prompt dissection and PPL filtering. Our results underscore the importance of prompt structure awareness and controlled perturbation for reliable adversarial robustness evaluation in LLMs. Code and data are available at https://github.com/Yujiaaaaa/PACP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。