从模型自身激活中挖掘可干预的行为技能,提升训练与推理效果。
Characterizing Model-Native Skills

- 从序列激活中提取模型原生的正交技能基,不依赖人类预设分类。
- 基于该基选择训练数据,数学推理准确率最高提升41%。
- 技能方向可直接用于推理时行为控制,实现人类无法定义的干预。
技能是描述语言模型能力及行为调控的自然单位。现有刻画方式依赖人工分类、文本描述或手动分析流程,这些外部假设未必与模型内部表征一致。我们主张,当目标是干预模型行为时,技能刻画应为‘模型原生’——基于模型自身的表征而非外部本体论强加。本文通过从序列级激活中恢复一个紧凑正交基,得到语义可解释但不对应任何预设人类分类的技能轴,捕捉了模型自身组织的行为变异方向。在推理后训练中验证,利用该基进行SFT数据筛选和推理时调制,均有效提升性能:在Llama3-8B与Qwen2.5-3B上,数据选择使MATH的Pass@1最高提升20%,AMC提升41%;推理阶段使用相同方向作为调制向量,使MATH的Pass@8最高提升4.8%,这是人类刻画技能无法支持的干预。在安全对齐任务中,基于模型原生技能覆盖而非文本多样性选择对抗训练数据,实现更高效的样本利用。结果表明,从模型内部表征中恢复技能,比外部强加更具干预有效性。代码已开源。
原文摘要 · Abstract (English)
Skills are a natural unit for describing what a language model can do and how its behavior can be changed. However, existing characterizations rely on human-written taxonomies, textual descriptions, or manual profiling pipelines--all external hypotheses about what matters that need not align with the model's internal representations. We argue that when the goal is to intervene on model behavior, skill characterization should be *model-native*: grounded in the model's own representations rather than imposed through external ontologies. We instantiate this view by recovering a compact orthogonal basis from sequence-level activations. The resulting basis is semantically interpretable but need not correspond to any predefined human ontology; instead, it captures axes of behavioral variation that the model itself organizes around. We validate this characterization on reasoning post-training, using the recovered basis for both SFT data selection and inference-time steering. We develop lightweight proxy interventions to identify which directions are most useful for a given model. Across Llama3-8B and Qwen2.5-3B, selecting data along those directions improves Pass@1 by up to 20% on MATH and 41% on AMC, outperforming data selection based on human-characterized skills. Because the basis lives in activation space, the same directions also serve as steering vectors at inference time, improving Pass@8 by up to 4.8% on MATH--an intervention that human-characterized skills cannot support. We further validate the characterization on safety alignment, where selecting adversarial training data for model-native skill coverage rather than textual diversity yields more sample-efficient learning. These results suggest that recovering skills from the model's own representations, rather than imposing them externally, provides a more effective foundation for intervening on model behavior. Codes are open-sourced.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。