arXiv:2509.16861cs.CRcs.AI2025-09中稿 · the ASE 2025 Inter…

提出可动态适应新攻击的智能防护系统,让大模型软件更安全。

AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Software

  • 通过持续学习识别新型越狱攻击,实现动态防御
  • 对未知攻击检测准确率达96%,两次更新即可适应新威胁
  • 适合部署后需持续防护的大模型应用开发者

大模型驱动的软件需要防护机制以确保安全。传统规则系统受限于预设输入输出空间,而大模型支持开放式交互,易受用户输入引发的越狱攻击。现有防护系统如LlamaGuard虽可达95%准确率,但在面对未见过的攻击时性能可能骤降至12%。为此,本文提出AdaptiveGuard,一种能检测未知越狱攻击(作为分布外输入)并基于持续学习框架动态适应的新防护机制。实证表明,AdaptiveGuard在分布外检测上达到96%准确率,仅需两次更新步骤即可适应新攻击,且在分布内数据上的F1分数仍保持85%以上,优于其他基线方法。结果证明该系统可在部署后持续进化以应对新兴越狱策略。相关代码与数据集已开源。

原文摘要 · Abstract (English)

Guardrails are critical for the safe deployment of Large Language Models (LLMs)-powered software. Unlike traditional rule-based systems with limited, predefined input-output spaces that inherently constrain unsafe behavior, LLMs enable open-ended, intelligent interactions--opening the door to jailbreak attacks through user inputs. Guardrails serve as a protective layer, filtering unsafe prompts before they reach the LLM. However, prior research shows that jailbreak attacks can still succeed over 70% of the time, even against advanced models like GPT-4o. While guardrails such as LlamaGuard report up to 95% accuracy, our preliminary analysis shows their performance can drop sharply--to as low as 12%--when confronted with unseen attacks. This highlights a growing software engineering challenge: how to build a post-deployment guardrail that adapts dynamically to emerging threats? To address this, we propose AdaptiveGuard, an adaptive guardrail that detects novel jailbreak attacks as out-of-distribution (OOD) inputs and learns to defend against them through a continual learning framework. Through empirical evaluation, AdaptiveGuard achieves 96% OOD detection accuracy, adapts to new attacks in just two update steps, and retains over 85% F1-score on in-distribution data post-adaptation, outperforming other baselines. These results demonstrate that AdaptiveGuard is a guardrail capable of evolving in response to emerging jailbreak strategies post deployment. We release our AdaptiveGuard and studied datasets at https://github.com/awsm-research/AdaptiveGuard to support further research.

大模型安全持续学习越狱防护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。