动态调节模型干预强度,避免无效修正,提升生成质量与可控性。
Dynamically Scaled Activation Steering
- 根据上下文自动调节干预强度,仅在需要时加强引导
- 结合现有方法后,在去毒与保持有用性上实现更好平衡
- 适用于文本与图像生成,可精准定位需干预的词元
激活引导已成为控制生成模型行为、如降低毒性输出的有效方法。但多数现有方法对所有输入统一施加干预,当无需引导时会损害模型性能。本文提出方法无关的动态缩放激活引导(DSAS),将何时引导与如何引导解耦。DSAS在层和输入间自适应调节现有引导变换的强度,仅在检测到不当行为时强干预。生成时,计算上下文相关的缩放因子,选择性调整任意引导方法的强度。我们还展示了可与引导函数端到端联合优化。结合现有方法后,DSAS持续提升帕累托前沿,在去毒与保用性之间取得更优权衡。进一步验证其通用性:应用于文本到图像扩散模型,实现对特定概念的自适应调控。最后,DSAS引入极小计算开销,同时提升可解释性,明确指出哪些词元需引导及程度。
原文摘要 · Abstract (English)
Activation steering has emerged as a powerful method for guiding the behavior of generative models towards desired outcomes such as toxicity mitigation. However, most existing methods apply interventions uniformly across all inputs, degrading model performance when steering is unnecessary. We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer. DSAS adaptively modulates the strength of existing steering transformations across layers and inputs, intervening strongly only when undesired behavior is detected. At generation time, DSAS computes context-dependent scaling factors that selectively adjust the strength of any steering method. We also show how DSAS can be jointly optimized end-to-end together with the steering function. When combined with existing steering methods, DSAS consistently improves the Pareto front with respect to steering alone, achieving a better trade-off between toxicity mitigation and utility preservation. We further demonstrate DSAS's generality by applying it to a text-to-image diffusion model, showing how adaptive steering allows the modulation of specific concepts. Finally, DSAS introduces minimal computational overhead while improving interpretability, pinpointing which tokens require steering and by how much.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。