通过短提示提前预警，让大模型在有害指令下自动停止推理。
SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment
- 用8个词的安全提示开头，触发模型自我约束
- 有害输出减少90%，拦截83.3%越狱攻击
- 轻量级设计，训练耗能仅为现有方法的1/300
大型推理模型（LRMs）虽能解决复杂问题，但在面对有害提示时可能产生危险输出。现有安全对齐方法虽可降低风险，但会削弱推理深度，且易被高级越狱攻击突破。为此，我们提出SAFEPATH，一种轻量级对齐方法：在有害提示下，模型会生成一个8词的安全提示（Safety Primer），从而提前终止推理，其余过程保持未监督。实验证明，SAFEPATH在多个基准上有效减少有害输出并维持推理性能，使DeepSeek-R1-Distill-Llama-8B模型的有害响应下降90.0%，成功阻断83.3%的越狱尝试，所需计算量仅为直接拒绝法的1/295.9和SafeChain的1/314.1。我们还提出了无需微调的零样本变体。此外，我们系统分析了现有LLM安全方法在推理型模型上的泛化能力，揭示关键缺陷与新方向。
原文摘要 · Abstract (English)
Large Reasoning Models (LRMs) have become powerful tools for complex problem solving, but their structured reasoning pathways can lead to unsafe outputs when exposed to harmful prompts. Existing safety alignment methods reduce harmful outputs but can degrade reasoning depth, leading to significant trade-offs in complex, multi-step tasks, and remain vulnerable to sophisticated jailbreak attacks. To address this, we introduce SAFEPATH, a lightweight alignment method that fine-tunes LRMs to emit a short, 8-token Safety Primer at the start of their reasoning, in response to harmful prompts, while leaving the rest of the reasoning process unsupervised. Empirical results across multiple benchmarks indicate that SAFEPATH effectively reduces harmful outputs while maintaining reasoning performance. Specifically, SAFEPATH reduces harmful responses by up to 90.0% and blocks 83.3% of jailbreak attempts in the DeepSeek-R1-Distill-Llama-8B model, while requiring 295.9x less compute than Direct Refusal and 314.1x less than SafeChain. We further introduce a zero-shot variant that requires no fine-tuning. In addition, we provide a comprehensive analysis of how existing methods in LLMs generalize, or fail, when applied to reasoning-centric models, revealing critical gaps and new directions for safer AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。