让大模型在推理中自检安全，自动防御越狱攻击
Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking
- 在生成过程中嵌入安全自检机制，每步推理都评估安全状态
- 引入安全锚点标记，使模型对越狱攻击的识别准确率显著提升
- 无需牺牲原有性能，适合需要高安全性的大模型应用
大型推理模型（LRMs）在多个领域表现出色，但其安全能力是否因推理能力增强而提升仍不明确。为此，本文提出一种名为 Reasoning-to-Defend（R2D）的新训练范式，将安全感知推理机制融入大语言模型（LLM）的生成过程。该机制在每一步推理中进行自我评估，生成安全锚点标记（safety pivot tokens）以指示响应的安全状态。为提升锚点预测准确性，进一步提出对比锚点优化（CPO），增强模型对对话安全状态的感知能力。模型在推理过程中动态调整响应策略，显著提升了抵御越狱攻击的能力。大量实验表明，R2D能有效缓解多种攻击，同时保持原有性能，凸显了安全感知推理在提升大模型鲁棒性方面的巨大潜力。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) have recently demonstrated impressive performances across diverse domains. However, how the safety of Large Language Models (LLMs) benefits from enhanced reasoning capabilities against jailbreak queries remains unexplored. To bridge this gap, in this paper, we propose Reasoning-to-Defend (R2D), a novel training paradigm that integrates a safety-aware reasoning mechanism into LLMs' generation process. This enables self-evaluation at each step of the reasoning process, forming safety pivot tokens as indicators of the safety status of responses. Furthermore, in order to improve the accuracy of predicting pivot tokens, we propose Contrastive Pivot Optimization (CPO), which enhances the model's perception of the safety status of given dialogues. LLMs dynamically adjust their response strategies during reasoning, significantly enhancing their safety capabilities defending jailbreak attacks. Extensive experiments demonstrate that R2D effectively mitigates various attacks and improves overall safety, while maintaining the original performances. This highlights the substantial potential of safety-aware reasoning in improving robustness of LRMs and LLMs against various jailbreaks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。