提出双阶段防御框架，提前拦截和中途阻断越狱攻击。
MetaDefense: Defending Finetuning-based Jailbreak Attack Before and During Generation
- 用特殊提示让模型预判问题和生成片段的危险性
- 在多个模型上对已知和未知攻击模板均有效
- 适合需要高安全性的大模型应用开发
本文提出MetaDefense，一种针对大语言模型（LLMs）微调型越狱攻击的新防御框架。现有防御机制难以泛化到未见过的攻击模板伪装的有害查询，尽管模型在嵌入空间中能识别此类查询。基于此，我们设计两阶段防御：(i) 生成前检测，提前发现有害查询；(ii) 生成中监控，实时检查部分输出以阻止有害内容扩散。MetaDefense通过专用提示训练模型预测查询与生成片段的有害性，实现潜在有害交互的早期终止。在多个模型架构（LLaMA-2-7B、Qwen-2.5-3B-Instruct、LLaMA-3.2-3B-Instruct）上的实验表明，该方法显著优于现有防御机制，在应对已知和未知攻击模板时均表现出强鲁棒性，同时保持良性任务上的良好性能。代码已开源。
原文摘要 · Abstract (English)
This paper introduces MetaDefense, a novel framework for defending against finetuning-based jailbreak attacks in large language models (LLMs). We observe that existing defense mechanisms fail to generalize to harmful queries disguised by unseen attack templates, despite LLMs being capable of distinguishing disguised harmful queries in the embedding space. Based on these insights, we propose a two-stage defense approach: (i) pre-generation defense that detects harmful queries before response generation begins, and (ii) mid-generation defense that monitors partial responses during generation to prevent outputting more harmful content. Our MetaDefense trains the LLM to predict the harmfulness of both queries and partial responses using specialized prompts, enabling early termination of potentially harmful interactions. Extensive experiments across multiple LLM architectures (LLaMA-2-7B, Qwen-2.5-3B-Instruct, and LLaMA-3.2-3B-Instruct) demonstrate that MetaDefense significantly outperforms existing defense mechanisms, achieving robust defense against harmful queries with seen and unseen attack templates while maintaining competitive performance on benign tasks. Code is available at https://github.com/ws-jiang/MetaDefense.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。