用稀疏自编码器净化小数据训练的指令向量，提升大模型控制效果。
Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement
- 通过稀疏自编码器识别并剔除无关语义特征
- 利用语义相似性补充小数据中缺失的关键特征
- 仅用少量数据即可生成高效指令向量，适合资源受限场景
指令控制已成为无需修改模型参数即可调控大语言模型的有效方法。然而，现有方法多依赖大规模数据学习清晰行为信息，限制了其在真实场景的应用。从少量数据中提取的指令向量常包含与任务无关的噪声特征，降低其有效性。为此，我们提出基于稀疏自编码器的指令向量精炼方法（SAE-RSV），利用SAEs对指令向量进行语义去噪和增强。框架首先根据SAEs提供的语义信息移除无关特征，再通过与已识别相关特征的语义相似性，补充小数据中缺失的任务相关特征。大量实验表明，所提方法显著优于所有基线方法，包括监督微调。研究发现，通过SAEs精炼原始指令向量，可仅用有限训练数据构建出高效的指令向量。
原文摘要 · Abstract (English)
Steering has emerged as a promising approach in controlling large language models (LLMs) without modifying model parameters. However, most existing steering methods rely on large-scale datasets to learn clear behavioral information, which limits their applicability in many real-world scenarios. The steering vectors extracted from small dataset often contain task-irrelevant noising features, which degrades their effectiveness. To refine the steering vectors learned from limited data, we introduce Refinement of Steering Vector via Sparse Autoencoder (SAE-RSV) that leverages SAEs to semantically denoise and augment the steering vectors. In our framework, we first remove task-irrelevant features according to their semantics provided by SAEs, and then enrich task-relevant features missing from the small dataset through their semantic similarity to the identified relevant features. Extensive experiments demonstrate that the proposed SAE-RSV substantially outperforms all the baseline methods including supervised fine-tuning. Our findings show that effective steering vector can be constructed from limited training data by refining the original steering vector through SAEs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。