通过细粒度激活调控,少干预却更精准地改变大模型行为。
Fine-Grained Activation Steering: Steering Less, Achieving More
- 按原子单元(AU)级别精细操控激活值,而非整块干预。
- 仅操纵有益的原子单元,效率更高且减少副作用。
- 适合需要高效微调大模型行为的研究者与工程师。
激活调控已成为低成本调整大语言模型(LLM)行为的有效方法。现有方法通常在模块层面干预,对选定注意力头、前馈网络或残差流的组合激活进行操作。然而我们发现,模块级激活本身具有内在异质性,混合了有益、无关和有害特征,导致模块级调控粗略、低效且具侵入性。为探究根源,我们将模块激活分解为细粒度的原子单元(AU)级激活,每个AU对应模块激活的一个维度,每个AU代表模块权重矩阵的一片区域。调控一个AU级激活等价于调控其关联的AU。理论与实证分析表明,异质性源于不同AU或维度控制输出中不同标记的分布。因此,模块级调控不可避免地同时移动有益与有害的标记方向,降低效率。仅限于有益的AU干预可实现更精确有效的调控。基于此,我们提出AUSteer,一种在原子单元级别操作的简单高效方法。首先通过对比样本计算激活动量,全局识别判别性AU;再针对不同输入和选择的AU分配自适应调控强度。在多个LLM和任务上的综合实验表明,AUSteer持续优于先进基线,同时调控的激活数量显著更少,证明‘少干预,多成效’。
原文摘要 · Abstract (English)
Activation steering has emerged as a cost-effective paradigm for modifying large language model (LLM) behaviors. Existing methods typically intervene at the block level, steering the bundled activations of selected attention heads, feedforward networks, or residual streams. However, we reveal that block-level activations are inherently heterogeneous, entangling beneficial, irrelevant, and harmful features, thereby rendering block-level steering coarse, inefficient, and intrusive. To investigate the root cause, we decompose block activations into fine-grained atomic unit (AU)-level activations, where each AU-level activation corresponds to a single dimension of the block activation, and each AU denotes a slice of the block weight matrix. Steering an AU-level activation is thus equivalent to steering its associated AU. Our theoretical and empirical analysis show that heterogeneity arises because different AUs or dimensions control distinct token distributions in LLM outputs. Hence, block-level steering inevitably moves helpful and harmful token directions together, which reduces efficiency. Restricting intervention to beneficial AUs yields more precise and effective steering. Building on this insight, we propose AUSteer, a simple and efficient method that operates at a finer granularity of the AU level. AUSteer first identifies discriminative AUs globally by computing activation momenta on contrastive samples. It then assigns adaptive steering strengths tailored to diverse inputs and selected AU activations. Comprehensive experiments on multiple LLMs and tasks show that AUSteer consistently surpasses advanced baselines while steering considerably fewer activations, demonstrating that steering less achieves more.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。