无需训练即可同时控制多个行为属性,提升生成内容一致性。
ORBIT: Training-Free Multi-Attribute Behavioral Steering via Orthogonal Subspace Rotation

- 通过正交子空间旋转统一多属性控制方向
- 在三个模型上实现更强且更平衡的多属性调节效果
- 适合需要灵活调控多种行为特征的智能助手场景
语言模型广泛应用于助手场景,行为属性控制至关重要。激活转向在推理时修改隐藏状态,是一种轻量级、无需训练的可动态切换机制。现有方法多仅支持单属性控制,当需同时调控多个属性时,简单叠加各属性转向向量会引发范数失衡和方向抵消;基于分类器的方法则在属性集变更时需重新训练。本文提出ORBIT(正交旋转干预技术),将基于旋转的转向扩展至多属性场景。该方法通过奇异值分解构建各属性转向平面的联合子空间,并在其中施加单一保范旋转,指向组合目标方向。自适应逐令牌门机制识别每位置需修正的属性,可选的加性增强模块强化初始投影弱的属性。我们还引入TraitFactory——一个聚焦行为倾向而非表层风格的新多属性基准。在TraitFactory与ToneBank上对Llama-3.2-3B、Qwen-2.5-7B、Llama-3.1-8B三模型进行多属性同时调控评估,结果表明,ORBIT在保持输出连贯性的同时,显著优于现有无需训练基线。
原文摘要 · Abstract (English)
Language models are widely used in assistant settings, where controlling behavioral attributes is often essential. Activation steering modifies hidden-state representations at inference time, providing a lightweight, training-free mechanism that can be toggled at runtime. Existing methods, however, have focused primarily on steering a single attribute at a time. When multiple attributes must be controlled simultaneously, naive summation of per-attribute steering vectors suffers from norm imbalance and directional cancellation, while classifier-based approaches require retraining whenever the attribute set changes. We introduce ORBIT (Orthogonal Rotation-Based Intervention Technique), a training-free extension of rotation-based steering to the multi-attribute setting. Our method constructs a joint subspace from per-attribute steering planes via singular value decomposition and applies a single norm-preserving rotation within that subspace toward a combined target direction. Adaptive per-token gating identifies which attributes need correction at each position, and an optional additive boost strengthens attributes with weak initial projection. We also introduce TraitFactory, a new multi-attribute benchmark that focuses on behavioral tendencies rather than surface-level style. We evaluate ORBIT on TraitFactory and ToneBank across three models (Llama-3.2-3B, Qwen-2.5-7B, Llama-3.1-8B) while steering multiple attributes simultaneously, showing that it achieves stronger and more balanced multi-attribute steering than existing training-free baselines while better preserving output coherence.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。