arXiv:2607.19364cs.AIcs.CL2026-07

无需优化即可透明控制大模型行为,通过统计共识筛选关键激活特征。

SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models

论文配图:SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models
图 1 · 摘自论文原文
  • 用六项可靠性条件过滤SAE特征,再通过三类统计量加权排序。
  • 在四个行为领域中优于七种基线方法,且保持生成质量与连贯性。
  • 适合关注可解释性与审计的AI安全、可控生成研究者使用。

激活引导在推理时添加残差流方向,实现无微调的轻量级行为控制。稀疏自编码器(SAEs)可通过将密集激活分解为近似单语义特征基,使干预过程可审计。本文提出SAE-StatSteer,一种透明且无需优化的流水线:首先通过六项可靠性条件筛选特征,再基于F检验、KSG互信息和Cohen's d三项统计量进行无权重布道共识排名,最后以Cohen's d权重组合选定的SAE解码器行。我们在三个Gemma系列模型上,针对四个行为领域对比了七种密集或SAE基基线。所提质量约束协议要求属性移动同时保持相关性、丰富性与连贯性。原始成功指标会高估可用控制,因强偏移常损害生成质量,且有效引导不依赖通用层或强度。SAE-StatSteer在性能上媲美优化方法,同时暴露每一步选择与加权决策供审计。结果支持报告质量约束后的成功率,而非仅原始行为偏移。代码与数据见https://github.com/Oshayer-Siddique/LLM-Steering-Using-SAE。

原文摘要 · Abstract (English)

Activation steering adds a residual-stream direction at inference time, providing lightweight behavioral control without fine-tuning. Sparse autoencoders (SAEs) can make such interventions auditable by decomposing dense activations into an approximately monosemantic feature basis. We introduce SAE-StatSteer, a transparent, optimization-free pipeline. It first filters features through six reliability conditions, then ranks the survivors by an unweighted Borda consensus over three statistics, an $F$-test, KSG mutual information, and Cohen's $d$, and finally combines the selected SAE decoder rows using Cohen's-$d$ weights. We evaluate three Gemma-family models across four behavioral domains against seven dense or SAE-based baselines. Our quality-conditioned protocol requires attribute movement while preserving relevance, richness, and coherence. Raw success systematically overstates usable control because strong shifts often degrade generation quality, and effective steering is not governed by a universal layer or strength. SAE-StatSteer remains competitive with optimization-based methods while exposing every selection and weighting decision for audit. These results motivate reporting quality-conditioned success alongside raw behavioral shift. Our code and data are available at https://github.com/Oshayer-Siddique/LLM-Steering-Using-SAE.

大模型控制激活引导可解释性SAE

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。