揭示语言模型激活调控中角度与幅值的分离作用，提升控制精度
A Geometric Account of Activation Steering through Angle-Norm Decomposition
- 通过角度-幅值分解，解耦隐藏状态的几何成分
- 发现概念主要由角度表示，但幅值影响调控稳定性
- 建议用可解释的双参数干预替代单一系数，适合模型调优者
线性激活调控因其简单有效而受到青睐。近期提出的球面调控范式旨在克服加法干预的局限，常基于隐藏状态幅值不携带语义信息的假设。本文通过受控实证研究，系统解耦角度与径向分量的作用。在七种语言模型上，我们发现概念主要编码于角度结构，支持球面方法动机；但幅值仍对调控稳定性和下游效果至关重要。结果解释了为何相似概念效果的干预表现不同，并表明激活调控应基于可解释的角度与径向分量进行参数化，而非使用纠缠两个效应的单一加法系数。
原文摘要 · Abstract (English)
Linear activation steering has gained popularity as a simple and empirically effective way to control language model behavior. More recently, spherical steering paradigms have been proposed to address limitations of additive interventions, often motivated by the assumption that hidden-state norm does not carry concept-relevant information. In this work, we revisit this assumption through a controlled empirical study designed to disentangle the roles of angular and radial components. We show that steering methods differ mainly in how they couple two geometric effects: changing a token's angular alignment with a concept direction and changing its hidden-state norm. Across seven language models, we find that concepts are represented primarily in angular structure, supporting the motivation for spherical methods, but that norm remains important for the stability and downstream effects of steering. Our results explain why interventions with similar concept-level effects can behave differently, and suggest that activation steering should be parameterized by interpretable angular and radial components of the intervention, rather than by a single additive coefficient that entangles these two effects.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。