让语言模型学会在概念分布层面精准控制行为，提升可控性与鲁棒性。
Learning Distribution-Wise Control in Representation Space for Language Models
- 在概念子空间的分布层面设计可学习干预，而非仅点级调整。
- 在8个常识推理和7个算术推理任务中，性能全面优于传统点级干预。
- 适合需要精细控制语言模型行为的研究者，尤其关注可控生成场景。
语言模型中的干预技术常用于前向传播中策略性地引导模型行为。可学习干预（又称表征微调）旨在概念子空间内实现点级控制，已被证明能有效改变高层行为。本文将该方法扩展至分布层面，使模型不仅能进行点级变换，还能学习概念子空间周围的区域。实验表明，该方法在早期层中表现优异，较大的标准差与性能提升显著相关。在8个常识推理和7个算术推理基准上，分布级干预在可控性和鲁棒性方面均持续优于点级干预。结果说明，分布级干预为语言模型行为调控提供了更全面的方法，支持更细粒度的控制。代码见：https://github.com/chili-lab/D-Intervention。
原文摘要 · Abstract (English)
Interventions in language models (LMs) are applied strategically to steer model behavior during the forward pass. Learnable interventions, also known as representation fine-tuning, aim to apply pointwise control within the concept subspace and have proven effective in altering high-level behaviors. In this work, we extend this approach to the distribution level, enabling the model to learn not only pointwise transformations but also the surrounding regions of the concept subspace. We demonstrate that these methods perform effectively in early layers, with larger standard deviations correlating strongly with improved performance. Across eight commonsense reasoning and seven arithmetic reasoning benchmarks, our distribution-wise interventions consistently outperform pointwise interventions in controllability and robustness. These results illustrate that distribution-wise interventions provide a more comprehensive method for steering model behavior and enabling finer-grained control over language models. The code is at: \href{https://github.com/chili-lab/D-Intervention}{https://github.com/chili-lab/D-Intervention}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。