提出新框架SPELL,检测大模型生成恶意代码的漏洞
Casting a SPELL: Sentence Pairing Exploration for LLM Limitation-breaking
- 用时间分段策略组合句子,智能构造越狱提示
- 在3个主流代码模型上最高达83.75%攻击成功率
- 适合研究AI安全与代码生成漏洞的开发者
大语言模型(LLMs)通过AI辅助编程工具革新了软件开发,使编程能力有限的开发者也能构建复杂应用。然而,这种可及性也使恶意行为者能利用强大工具生成有害软件。现有越狱研究多聚焦于通用攻击场景,对恶意代码生成作为越狱目标的探索有限。为此,我们提出SPELL——一个专门评估恶意代码生成中安全对齐弱点的综合性测试框架。该框架采用时间分段选择策略,通过智能组合先验知识数据集中的句子,系统化构建越狱提示,在探索新型攻击模式与复用成功技术之间取得平衡。在GPT-4.1、Claude-3.5和Qwen2.5-Coder三个先进代码模型上进行的广泛评估表明,SPELL在八类恶意代码中分别达到83.75%、19.38%和68.12%的攻击成功率。生成的提示能在真实AI开发工具Cursor中输出恶意代码,且经最先进的检测系统确认为恶意的比例超过73%。这些发现揭示了当前LLM实现中的显著安全缺口,为提升代码生成应用中的AI安全对齐提供了宝贵洞见。
原文摘要 · Abstract (English)
Large language models (LLMs) have revolutionized software development through AI-assisted coding tools, enabling developers with limited programming expertise to create sophisticated applications. However, this accessibility extends to malicious actors who may exploit these powerful tools to generate harmful software. Existing jailbreaking research primarily focuses on general attack scenarios against LLMs, with limited exploration of malicious code generation as a jailbreak target. To address this gap, we propose SPELL, a comprehensive testing framework specifically designed to evaluate the weakness of security alignment in malicious code generation. Our framework employs a time-division selection strategy that systematically constructs jailbreaking prompts by intelligently combining sentences from a prior knowledge dataset, balancing exploration of novel attack patterns with exploitation of successful techniques. Extensive evaluation across three advanced code models (GPT-4.1, Claude-3.5, and Qwen2.5-Coder) demonstrates SPELL's effectiveness, achieving attack success rates of 83.75%, 19.38%, and 68.12% respectively across eight malicious code categories. The generated prompts successfully produce malicious code in real-world AI development tools such as Cursor, with outputs confirmed as malicious by state-of-the-art detection systems at rates exceeding 73%. These findings reveal significant security gaps in current LLM implementations and provide valuable insights for improving AI safety alignment in code generation applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。