让大模型按输入动态选择最佳激活层,提升角色控制效果且不损失流畅性。
Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models

- 根据输入动态选择多层激活注入位置,而非固定全局层
- 在80亿参数模型上实现接近最优策略的性能,避免过度调整导致输出崩塌
- 部署时仅需提示词,无需标签或额外计算,适合实际应用
激活调优通过向冻结语言模型的残差流中添加学习向量来改变其行为,现有方法对每项任务固定全局注入层。本文认为最佳层应为实例级决策,提出可理解且可部署的逐实例多层选择方法。在两个开放权重的80亿参数模型与六个二元人格特质上,逐实例最优层组合显示层选择随输入变化:多数特征-模型组合中,固定全局层无法恢复逐实例优势。一种基于单层边际效应排序的贪心规则几乎复现了最优收益,但需对比真实答案,无法用于部署;因此将其作为目标,训练一个仅依赖提示词的预测器来复现该规则。所提部署方案无需推理标签:从提示嵌入中读取逐实例层排名,通过分类器推断调优方向,并用自适应门控评估短路调优结果,仅当必要时才增加更多层。该方案在强模型上恢复了大部分最优收益,在弱模型上也取得明显优势,平均不低于原始对齐基线,且显著缓解了高层数全局调优引发的流畅性下降问题。机制分析‘方向优于幅度’解释了错误全局调优导致的行为反转、过度调优引起的输出崩溃及不可调优输入的上限现象。
原文摘要 · Abstract (English)
Activation steering edits the behaviour of a frozen language model by adding a learned vector to its residual stream, and current practice fixes the injection layers globally per task. We argue that the best layers are an instance-level decision, and we make per-instance, multi-layer selection both well understood and deployable. On two open-weight 8B models and six binary persona traits, a per-instance oracle over layer subsets shows that the best layers vary from one input to the next: on most trait-model pairs, no fixed global layer set recovers the per-instance benefit. A greedy rule that ranks layers by single-layer marginal effect recovers nearly all of the oracle's benefit, but both must score candidate layers against the gold answer, so neither can run at deployment; the rule instead becomes the target a prompt-only predictor is trained to reproduce. Our deployable recipe needs no label at inference: a per-instance layer ranker read off the prompt embedding, a classifier that infers the steering direction, and an adaptive gate that scores short steered passes against that inferred direction and steers no more layers than necessary. The recipe recovers most of the oracle's lift (the bulk on the stronger model, a clear majority on the harder one), never drives any trait-model pair below its unsteered alignment baseline on average, and largely avoids the fluency collapse that strong global selection incurs at higher layer counts. A mechanistic account, "direction over magnitude", explains the behavioural flip under a mis-directed global set, the output collapse from steering too many layers, and the ceiling of unsteerable inputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。