通过缩放激活向量，实现对语言模型的精准干预与解释。
Activation Scaling for Steering and Interpreting Language Models
- 仅通过缩放少数激活向量的符号幅度实现模型干预。
- 在合成任务中效果媲美传统向量，且更稀疏高效。
- 可学习为激活向量的函数，适配不同长度提示。
给定提示“Rome is in”，能否仅通过缩放若干相关激活向量的标量，使语言模型将错误预测的“France”修正为正确答案“Italy”？我们认为成功干预是理解模型内部机制的前提。具体而言，我们提出一个三重目标：有效干预应交换正确与错误词的预测（有效性），同时保持其他词不变（忠实性），且干预应尽可能稀疏（最小性）。基于梯度优化，该目标可学习并评估一种高效、可解释的干预方式：激活缩放仅调整激活向量的符号幅度，以增强、削弱或反转模型内已编码的引导方向。在合成任务中，该方法在有效性和忠实性上可媲美传统引导向量，但显著更稀疏，有助于定位可解释的模型组件。我们从多角度评估激活缩放，对比不同数据集表现，并将缩放系数设为激活向量自身的可学习函数，以适应不同长度的提示。
原文摘要 · Abstract (English)
Given the prompt "Rome is in", can we steer a language model to flip its prediction of an incorrect token "France" to a correct token "Italy" by only multiplying a few relevant activation vectors with scalars? We argue that successfully intervening on a model is a prerequisite for interpreting its internal workings. Concretely, we establish a three-term objective: a successful intervention should flip the correct with the wrong token and vice versa (effectiveness), and leave other tokens unaffected (faithfulness), all while being sparse (minimality). Using gradient-based optimization, this objective lets us learn (and later evaluate) a specific kind of efficient and interpretable intervention: activation scaling only modifies the signed magnitude of activation vectors to strengthen, weaken, or reverse the steering directions already encoded in the model. On synthetic tasks, this intervention performs comparably with steering vectors in terms of effectiveness and faithfulness, but is much more minimal allowing us to pinpoint interpretable model components. We evaluate activation scaling from different angles, compare performance on different datasets, and make activation scalars a learnable function of the activation vectors themselves to generalize to varying-length prompts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。