揭示大模型各层稀疏自编码器特征的演化规律。
Evolution of SAE Features Across Layers in LLMs
- 分析相邻层特征间的统计关联，构建跨层特征图谱。
- 发现部分特征可由前层特征组合而成，部分在深层更专精。
- 适合研究模型内部表征演化的研究人员参考。
基于Transformer的语言模型通常在每层独立定义稀疏自编码器。本文分析了相邻层特征间的统计关系，以理解特征在前向传播过程中的演化。我们提供了一个特征及其最相似下一层邻居的可视化界面（https://stefanhex.com/spar-2024/feature-browser/），并构建了跨层相关特征社区。结果表明，相当一部分特征会从前层传递到后层，某些特征可表示为前层特征的准布尔组合，且部分特征在深层变得更加专精。
原文摘要 · Abstract (English)
Sparse Autoencoders for transformer-based language models are typically defined independently per layer. In this work we analyze statistical relationships between features in adjacent layers to understand how features evolve through a forward pass. We provide a graph visualization interface for features and their most similar next-layer neighbors (https://stefanhex.com/spar-2024/feature-browser/), and build communities of related features across layers. We find that a considerable amount of features are passed through from a previous layer, some features can be expressed as quasi-boolean combinations of previous features, and some features become more specialized in later layers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。