通过精准选层实现无损可控的模型行为调控
Selective Steering: Norm-Preserving Control Through Discriminative Layer Selection
- 基于判别性层选择与保范旋转，实现稳定控制
- 攻击成功率提升5.5倍，困惑度零违规，能力保留率约100%
- 适合需要安全可控推理的模型部署场景
尽管对齐技术取得进展，大语言模型仍易受诱发有害行为的对抗攻击。激活引导方法虽在推理时提供干预可能，但现有方法存在明显缺陷：激活加法需精细调节系数且对层间范数变化敏感，方向性删除仅支持二值控制。近期提出的角向引导通过二维子空间旋转实现连续控制，但其实现方式破坏了范数保持，引发分布偏移与生成崩溃，尤其在参数量低于70亿的模型中更为显著。本文提出选择性引导（Selective Steering），通过两项关键创新解决上述问题：(1) 数学严谨的保范旋转公式，确保激活分布完整性；(2) 判别性层选择机制，仅在特征表示呈现反向类别对齐的层上施加引导。九个模型的实验表明，该方法相比先前方法攻击成功率提升5.5倍，同时维持零困惑度违规，并在标准基准上实现约100%的能力保留。本方法为可控、稳定的语言模型行为修改提供了原理清晰且高效的框架。
原文摘要 · Abstract (English)
Despite significant progress in alignment, large language models (LLMs) remain vulnerable to adversarial attacks that elicit harmful behaviors. Activation steering techniques offer a promising inference-time intervention approach, but existing methods suffer from critical limitations: activation addition requires careful coefficient tuning and is sensitive to layer-specific norm variations, while directional ablation provides only binary control. Recent work on Angular Steering introduces continuous control via rotation in a 2D subspace, but its practical implementation violates norm preservation, causing distribution shift and generation collapse, particularly in models below 7B parameters. We propose Selective Steering, which addresses these limitations through two key innovations: (1) a mathematically rigorous norm-preserving rotation formulation that maintains activation distribution integrity, and (2) discriminative layer selection that applies steering only where feature representations exhibit opposite-signed class alignment. Experiments across nine models demonstrate that Selective Steering achieves 5.5x higher attack success rates than prior methods while maintaining zero perplexity violations and approximately 100\% capability retention on standard benchmarks. Our approach provides a principled, efficient framework for controllable and stable LLM behavior modification. Code: https://github.com/knoveleng/steering
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。