通过自洽推理轨迹实现更稳定的模型激活调控，提升大模型推理性能。
ROAST: Rollout-based On-distribution Activation Steering Technique
- 基于模型自身推理过程生成的分布内轨迹估计激活方向
- 在多个模型上提升任务表现，最高增益达12.1%
- 采用连续软缩放与分组归一化，避免激活值失衡
激活调控可在推理时以参数高效方式控制大语言模型，但许多方法依赖分布外监督和离散掩码，导致干预结果脆弱。本文提出ROAST（基于推理轨迹的分布内激活调控技术），通过模型自身的分布内推理轨迹估计调控方向，利用ROC方法并采用连续软缩放（CSS）和分组均值归一化避免硬稀疏化。实证分析显示，激活幅值虽与方向一致性中度相关，但其方差显著且常与语义质量不成比例，高幅值激活若不归一化可能主导全局调控方向。为此，ROAST引入分组归一化平衡不同样本贡献，确保共识调控方向更稳健。在0.6B至32B规模的多个模型上，ROAST持续提升多样任务性能（如Qwen3-0.6B在GSM8K上提升9.7%，GLM4-32B在TruthfulQA上提升12.1%），分析表明CSS更有效保留激活能量。
原文摘要 · Abstract (English)
Activation steering provides parameter-efficient control over large language models (LLMs) at inference time, but many methods rely on off-distribution supervision and discrete masking, leading to brittle interventions. We propose ROAST (Rollout-based On-distribution Activation Steering Technique), which estimates steering directions from the model's own on-distribution rollouts via ROC and avoids hard sparsification via Continuous Soft Scaling (CSS) and Grouped Mean Normalization. Our empirical analysis reveals that while activation magnitude correlates moderately with directional consistency, the variance in magnitude is significant and often disproportionate to semantic quality. This suggests that high-magnitude activations risk dominating the global steering direction if not properly normalized. To address this, ROAST employs grouped normalization to balance contributions across samples, ensuring a more robust estimation of the consensus steering direction. Across models (0.6B to 32B), ROAST consistently improves performance on diverse tasks (e.g., +9.7% on GSM8K for Qwen3-0.6B and +12.1% on TruthfulQA for GLM4-32B), and analyses show that CSS better preserves activation energy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。