通过激活方向控制代码模型的语言和库偏好，让其按需生成指定代码。
Steering Code LLMs with Activation Directions for Language and Library Control
- 用激活空间中的方向向量引导模型生成特定语言/库代码。
- 中性提示下目标生态生成率显著提升，强干预仍有效。
- 适合需要精准控制代码风格的开发者与研究者使用。
代码大模型在中性提示下常默认生成特定编程语言和库的代码。我们探究这些偏好是否以近似线性方向编码在激活空间中，并可在推理时操控。通过均值差法，我们为五组语言/库组合估计了逐层引导向量，并在生成过程中将其加至模型隐藏状态。在三款开源代码大模型上，该方法显著提升了中性提示下对目标生态的生成比例，且在提示明确要求相反选择时仍有效。引导强度因模型和目标而异，常见生态较易诱导，过强干预会降低输出质量。结果表明，代码风格偏好部分由激活空间中紧凑可调的结构表征。
原文摘要 · Abstract (English)
Code LLMs often default to particular programming languages and libraries under neutral prompts. We investigate whether these preferences are encoded as approximately linear directions in activation space that can be manipulated at inference time. Using a difference-in-means method, we estimate layer-wise steering vectors for five language/library pairs and add them to model hidden states during generation. Across three open-weight code LLMs, these interventions substantially increase generation toward the target ecosystem under neutral prompts and often remain effective even when prompts explicitly request the opposite choice. Steering strength varies by model and target, with common ecosystems easier to induce than rarer alternatives, and overly strong interventions can reduce output quality. Overall, our results suggest that code-style preferences in LLMs are partly represented by compact, steerable structure in activation space.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。