用安全论文生成攻击提示，让大模型失效
Paper Summary Attack: Jailbreaking LLMs through LLM Safety Papers
- 从安全论文中提取内容合成攻击模板
- 对齐模型攻击成功率高达97%
- 不同模型对攻击/防御论文反应迥异
大型语言模型（LLMs）的安全性受到广泛关注。本文指出，已有实证研究表明，LLMs倾向于信任权威来源的信息，如学术论文，这可能带来新漏洞。为验证此猜想，我们设计初步分析，得出两项发现。基于此，提出新型越狱方法——论文摘要攻击（Paper Summary Attack, PSA），系统性整合攻击型或防御型的LLM安全论文内容，构建对抗性提示模板，并在预设章节中嵌入有害查询作为对抗载荷。大量实验表明，该方法不仅在基础模型中有效，且在Deepseek-R1等先进推理模型上同样显著。PSA在对齐模型Claude3.5-Sonnet上达到97%的攻击成功率（ASR），在Deepseek-R1上更达98%。更有趣的是，不同基础模型间，甚至同一模型的不同版本间，面对攻击型或防御型论文时表现出截然相反的脆弱性偏差。这一现象或为未来对抗方法与安全对齐研究提供新线索。代码已开源：https://github.com/233liang/Paper-Summary-Attack
原文摘要 · Abstract (English)
The safety of large language models (LLMs) has garnered significant research attention. In this paper, we argue that previous empirical studies demonstrate LLMs exhibit a propensity to trust information from authoritative sources, such as academic papers, implying new possible vulnerabilities. To verify this possibility, a preliminary analysis is designed to illustrate our two findings. Based on this insight, a novel jailbreaking method, Paper Summary Attack (\llmname{PSA}), is proposed. It systematically synthesizes content from either attack-focused or defense-focused LLM safety paper to construct an adversarial prompt template, while strategically infilling harmful query as adversarial payloads within predefined subsections. Extensive experiments show significant vulnerabilities not only in base LLMs, but also in state-of-the-art reasoning model like Deepseek-R1. PSA achieves a 97\% attack success rate (ASR) on well-aligned models like Claude3.5-Sonnet and an even higher 98\% ASR on Deepseek-R1. More intriguingly, our work has further revealed diametrically opposed vulnerability bias across different base models, and even between different versions of the same model, when exposed to either attack-focused or defense-focused papers. This phenomenon potentially indicates future research clues for both adversarial methodologies and safety alignment.Code is available at https://github.com/233liang/Paper-Summary-Attack
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。