arXiv:2606.11599cs.CLcs.LG2026-06被引 1

用模型初期状态预测大模型可控性,省去大量无效尝试。

When is Your LLM Steerable?

论文配图:When is Your LLM Steerable?
图 1 · 摘自论文原文
  • 通过分析生成前几字的隐藏状态,判断控制指令能否生效
  • 预测准确率达0.7宏F1,可提前避免失败的控制尝试
  • 适合需要高效调参的工程落地场景

激活控制为推理阶段调节语言模型行为提供了一种轻量方法,但其成功与否高度依赖于提示、概念、模型和控制配置。确定有效控制范围通常需耗费大量资源进行网格搜索和完整自回归推演后评估。本文研究是否可通过生成初期的模型内部状态(如生成前几个词后)预测控制效果,并利用该预测器提升控制成功率。为此,我们构建了包含140万次控制生成的测试平台ASTEER,涵盖150个概念,每个样本标注了控制成功或失败。基于此平台,我们提取各层及初始解码步中控制前后隐藏状态的对比特征,分析控制效应在层间与词位置间的传播规律,揭示其对可控制性的关键作用。随后,我们在这些特征上训练梯度提升决策树(GBDT)分类器,实现无需完整推演即可预测干预将导致欠控、成功或过控。该预测器在未见概念上达到约0.7的宏平均F1得分,表明早期隐藏状态蕴含丰富且结构化的控制有效性信息。进一步地,我们将该预测器用于指导控制强度搜索,仅用少量解码成本即获得接近最优性能。

原文摘要 · Abstract (English)

Activation steering offers a lightweight approach to control language models' behavior at inference time, but whether it succeeds or fails heavily depends on the prompt, concept, model, and steering configuration. Finding the regime and boundaries of successful steering typically requires expensive grid searches and post-hoc evaluation of full autoregressive rollouts. In this work, we investigate whether steerability can be predicted from the model's internal states at the beginning of the generation process, e.g., after generating the first few tokens, and how to leverage such a predictor to improve steering success rate. To this end, we first introduce ASTEER, a testbed including 1.4M steered generations, spanning 150 concepts with each steering success/failure labeled. Leveraging this testbed, we analyze the model's early decoding dynamics by extracting features that compare hidden states before and after steering across layers and initial decoding steps. These features help us understand how steering's effects propagate along layers and token positions, which provide key information for steerability prediction. We then train a Gradient Boosting Decision Trees (GBDT) classifier on these features to predict whether an intervention will under-steer, succeed, or over-steer without requiring full rollout. Our predictor achieves around 0.7 macro-F1 score on unseen concepts, demonstrating that early hidden states encode substantial, structured information about eventual steering efficacy. We further leverage this steerability predictor as guidance for steering strength searching, achieving near-optimal performance with a small fraction of decoding cost.

模型控制推理优化预测机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。