让AI在拒绝有害请求时给出有用替代方案，避免机械拒绝对话
ALTSTEER: Selective Safety Steering for Moving Beyond Hard Refusals to Constructive Alternatives
- 基于内部拒绝信号判断何时干预，实现精准安全控制
- 通过分阶段引导，将僵硬拒绝转为建设性安全回应
- 无需训练，在Llama-3.1和Qwen2.5上显著提升安全响应质量
安全对齐对大模型部署至关重要，需在阻止有害响应的同时保持对良性请求的帮助性。激活控制提供了一种无需训练的推理阶段安全控制方法，但有效安全引导仍需解决两个耦合问题：何时干预，以及干预后如何引导生成。现有方法在两方面均受限：触发机制跨领域不稳定，以拒绝为导向的控制常导致僵硬拒绝而非建设性指导。为此，我们提出ALTSTEER，一种推理阶段框架，将选择性干预与拒绝锚定的建设性重定向结合于单一推理过程。ALTSTEER利用内部拒绝相关信号决定是否干预，并采用分阶段引导策略，将生成从拒绝导向转向建设性替代。在Llama-3.1和Qwen2.5上的评估显示，ALTSTEER在保留良性任务帮助性的同时，显著提升了建设性安全完成行为，尤其在原本倾向于对有害请求给出简短拒绝的模型上效果更优。
原文摘要 · Abstract (English)
Safety alignment is essential for deploying large language models, requiring systems to prevent harmful compliance while preserving helpfulness on benign requests. Activation steering offers a training-free inference-time approach to safety control, but effective safety steering requires addressing two coupled questions: when to intervene and how generation should be shaped after intervention. However, existing safety steering methods remain limited along both dimensions, as their triggering mechanisms can be unstable across domains and refusal-oriented steering often yields rigid refusals rather than constructive safe guidance. To address these limitations, we propose ALTSTEER, an inference-time framework that couples selective intervention with refusal-anchored constructive redirection within a single inference pass. ALTSTEER uses an internal refusal-relevant signal to decide when to steer, and applies staged steering to shift generation from refusal-oriented control toward constructive alternatives. Evaluations on Llama-3.1 and Qwen2.5 show that ALTSTEER preserves benign utility while improving constructive safe-completion behavior, especially on models that otherwise tend to produce short refusals for harmful requests.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。