提出新型查询自适应激活调控方法,提升大模型推理控制效果
Conditional Optimal Bridge for Riemannian Activation Steering

- 基于黎曼流形上的薛定谔桥建模,从优化问题推导出可解释的调控方向
- 在4个模型、3类对齐任务中均超越现有方法,且避免分布外输入性能下降
- 适合需要精准、自适应控制大模型行为的研究者和开发者
激活调控为大语言模型推理时的轻量级控制提供了替代微调的方法。尽管现有方法隐式优化目标激活与非目标激活分布之间的对数密度比,但其优化过程是启发式的,并未从严格优化问题推导而来。此外,这些方法生成的调控方向与查询无关,导致在分布内和分布外输入上性能下降。本文提出 extsc{Cobras}(条件最优桥的黎曼激活调控),将激活调控建模为残差流超球面上的薛定谔桥。该公式首次从一个良好定义的优化问题中推导出对数密度比调控目标。通过熵正则最优传输求解桥结构,并提取概率流微分方程,当Sinkhorn势函数均匀时,可还原出广泛使用的密度比梯度作为特例。关键在于,薛定谔势在当前激活点计算,使调控方向天然具备查询自适应性。实验证明,在4个模型和3条对齐轴(帮助性、真实性、去毒化)上, extsc{Cobras} 均一致优于现有激活调控基线,且避免了常见于现有方法的分布外性能退化。代码见 https://github.com/arshandalili/cobras。
原文摘要 · Abstract (English)
Activation steering offers a lightweight alternative to fine-tuning for controlling large language models at inference time. While many existing methods implicitly optimize a log-density-ratio objective between desired and undesired activation distributions, they do so heuristically rather than deriving it from a principled optimization problem. Moreover, these methods produce query-independent steering directions that can degrade performance on both in-distribution and out-of-distribution (OOD) inputs. We introduce \textsc{Cobras} (Conditional Optimal Bridge for Riemannian Activation Steering), which addresses both limitations by casting activation steering as a Schrödinger Bridge on the residual-stream hypersphere. This formulation yields, to our knowledge, the first principled derivation of the log-density-ratio steering objective from a well-posed optimization problem. Solving the bridge via entropic optimal transport and extracting the probability flow ODE recovers the widely used density-ratio gradient as a special case when the Sinkhorn potentials are uniform. Crucially, the Schrödinger potentials are evaluated at the current activation, making the resulting steering direction inherently query-adaptive. Empirically, across four models and three alignment axes (helpfulness, truthfulness, and detoxification), \textsc{Cobras} consistently outperforms prior activation steering baselines while avoiding the OOD degradation commonly observed in existing methods. The code can be found at https://github.com/arshandalili/cobras.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。