通过微调与输入净化结合，有效抵御长上下文越狱攻击。
Mitigating Many-Shot Jailbreaking
- 采用微调+输入净化双重策略应对越狱攻击
- 组合方法显著降低攻击成功率，保持正常任务性能
- 适合关注大模型安全防护的研究者与开发者
多示例越狱（MSJ）是一种对抗性技术，利用现代大模型的长上下文窗口，在提示中插入多个伪造助手的不当响应示例，再提出最终请求。当示例足够多时，模型的上下文学习能力会覆盖其安全训练，导致模型表现出伪造助手的行为。本文探究了不同微调和输入净化方法在缓解此类攻击上的效果，单独及组合使用。结果表明，各方法均有渐进式缓解效果，联合应用可显著降低MSJ攻击有效性，同时保持模型在良性上下文学习和对话任务中的表现。建议将该方法纳入模型安全后训练流程，以实质性缓解此漏洞。
原文摘要 · Abstract (English)
Many-shot jailbreaking (MSJ) is an adversarial technique that exploits the long context windows of modern LLMs to circumvent model safety training by including in the prompt many examples of a "fake" assistant responding inappropriately before the final request. With enough examples, the model's in-context learning abilities override its safety training, and it responds as if it were the "fake" assistant. In this work, we probe the effectiveness of different fine-tuning and input sanitization approaches on mitigating MSJ attacks, alone and in combination. We find incremental mitigation effectiveness for each, and show that the combined techniques significantly reduce the effectiveness of MSJ attacks, while retaining model performance in benign in-context learning and conversational tasks. We suggest that our approach could meaningfully ameliorate this vulnerability if incorporated into model safety post-training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。