用大模型自动生成稀疏编码器特征的自然语言解释。
SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization
- 将编码器方向注入大模型，微调后生成特征解释。
- 能泛化到未见过的特征，跨字典迁移有效。
- 适合需要可解释性的大模型研究与调试者。
稀疏自编码器（SAE）用于从大语言模型（LLM）表示中提取大量特征，但当前对这些特征的解释仍主要依赖外部观测，导致解释表面化且大规模收集行为证据效率低下。我们提出SAEVerbalizer框架，将SAE解码器方向注入LLM表示，并微调其下游层以生成自然语言形式的特征解释。训练完成后，该生成器可直接从解码器方向解释SAE特征，同时解决上述双重问题。实验表明，学习到的解释能力可泛化至未见特征，跨独立训练的SAE字典实现迁移，并通过轻量适配器扩展至不同LLM的SAE特征。干预实验显示，注入多个方向可生成组合语义的解释，而反转单个方向则产生相应语义偏移。
原文摘要 · Abstract (English)
Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale. We introduce SAEVerbalizer, a framework that injects SAE decoder directions into an LLM's representations and fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both limitations. Our experiments show that the learned verbalization capability generalizes to unseen features, transfers across separately trained SAE dictionaries, and, with a lightweight adapter, extends to SAE features from different LLMs. Intervention experiments show that injecting multiple directions yields an explanation combining their meanings, while reversing individual directions produces corresponding meaning shifts.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。