用对话轮次平均激活，让模型解释长文本更高效。
Turn-Averaged SAEs for Feature Discovery and Long-Context Attribution
- 对每轮对话的激活值取平均，固定特征数量
- 轮次级特征比单个词特征更完整描述对话内容
- 适合研究长对话中的语义归属与模式发现
稀疏自编码器（SAEs）已成为提取语言模型可解释特征的有效工具。但传统SAE在单个标记激活上运行，导致活跃特征数随上下文长度线性增长，难以分析长文本。本文提出轮次平均SAE，通过学习重建每轮（人类或助手）的平均模型激活，以固定数量特征表示单轮内容。实验表明，由LLM评估，轮次平均特征比逐标记特征更全面地描述单轮高层特征。同时，轮次平均SAE显著简化了下游任务如归因图的构建。总体而言，该方法使可解释性技术在长上下文场景中变得可行。
原文摘要 · Abstract (English)
Sparse autoencoders (SAEs) have become a useful tool for extracting interpretable features in language models. However, standard SAE architectures operate on individual token activations, meaning that the number of active features scales linearly with context length, and studying long model transcripts becomes difficult. We introduce turn-averaged SAEs, which represent a single Human or Assistant turn with a fixed number of features by learning to reconstruct the average model activation across the turn. We find that turn-averaged features describe a single turn's high-level characteristics more completely than per-token features when judged by an LLM. We also demonstrate that turn-averaged SAEs greatly simplify common downstream uses of SAEs like attribution graphs. Broadly, turn-averaged SAEs make interpretability techniques practical at long context lengths.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。