arXiv:2602.05234cs.LGcs.CL2026-02中稿 · ICLR

通过分布匹配实现双向精准模型操控,提升可控性与稳定性。

Faithful Bi-Directional Model Steering via Distribution Matching and Distributed Interchange Interventions

  • 用分布匹配替代概率最大化,弱监督学习干预策略
  • 基于分布式交换干预实现双向控制,减少超参调优
  • 在安全任务中有效覆盖拒答行为,保持模型通用能力

基于干预的模型操控为提示和微调提供了轻量且可解释的替代方案。然而,现有方法因沿用微调中的强优化目标,易过拟合且表现不佳,常生成不自然输出。我们假设有效操控需精准识别模型内部机制,而非强制外部偏好。为此,我们基于分布式对齐搜索(DAS)原理,提出新方法:概念DAS(CDAS)。虽采用DAS的核心机制——分布式交换干预(DII),但引入针对操控任务设计的分布匹配目标,使干预后输出分布与反事实分布对齐。CDAS主要区别于以往工作:其一,通过弱监督分布匹配学习干预,而非概率最大化;其二,利用天然支持双向操控的DII,使操控因子可从数据中推导,降低超参调优成本,提升控制精度与稳定性。在AxBench大规模操控基准上,CDAS未必始终优于偏好优化方法,但在模型规模增大时受益更明显。在两个安全相关案例中——覆盖安全对齐模型的拒答行为、消除思维链后门——CDAS实现了系统性操控并维持了模型通用能力。结果表明,CDAS与偏好优化方法互补,条件性地构成一种稳健的基于干预的模型操控方案。代码已开源:https://github.com/colored-dye/concept_das。

原文摘要 · Abstract (English)

Intervention-based model steering offers a lightweight and interpretable alternative to prompting and fine-tuning. However, by adapting strong optimization objectives from fine-tuning, current methods are susceptible to overfitting and often underperform, sometimes generating unnatural outputs. We hypothesize that this is because effective steering requires the faithful identification of internal model mechanisms, not the enforcement of external preferences. To this end, we build on the principles of distributed alignment search (DAS), the standard for causal variable localization, to propose a new steering method: Concept DAS (CDAS). While we adopt the core mechanism of DAS, distributed interchange intervention (DII), we introduce a novel distribution matching objective tailored for the steering task by aligning intervened output distributions with counterfactual distributions. CDAS differs from prior work in two main ways: first, it learns interventions via weak-supervised distribution matching rather than probability maximization; second, it uses DIIs that naturally enable bi-directional steering and allow steering factors to be derived from data, reducing the effort required for hyperparameter tuning and resulting in more faithful and stable control. On AxBench, a large-scale model steering benchmark, we show that CDAS does not always outperform preference-optimization methods but may benefit more from increased model scale. In two safety-related case studies, overriding refusal behaviors of safety-aligned models and neutralizing a chain-of-thought backdoor, CDAS achieves systematic steering while maintaining general model utility. These results indicate that CDAS is complementary to preference-optimization approaches and conditionally constitutes a robust approach to intervention-based model steering. Our code is available at https://github.com/colored-dye/concept_das.

模型操控因果干预安全对齐分布匹配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。