arXiv:2605.06494cs.AI2026-05

用图结构分析稀疏自编码器特征的共现模式,发现新类型语义模板。

From Token Lists to Graph Motifs: Weisfeiler-Lehman Analysis of Sparse Autoencoder Features

  • 将每个特征表示为上下文中共现的词元构成的图
  • 识别出标点密集、语言/脚本分组和代码模板等结构模式
  • 适合研究模型内部语义结构的可解释性研究人员

稀疏自编码器(SAEs)已成为机制可解释性的核心工具,能将Transformer激活分解为单义特征。然而现有分析主要依赖最高激活词元列表或解码器权重向量,忽略了特征间共享的高阶共现结构。本文提出一种图结构表示:每个SAE特征被建模为一个词元共现图,节点为强激活附近最频繁出现的词元,边连接局部上下文窗口内共现的词元对。采用定制的类Weisfeiler-Lehman、按频率分箱的图核计算该结构空间的相似性。以在GPT-2 Small上训练的大规模SAE为例,结合合成混合领域语料探查,聚类结果恢复出启发式基序族(如标点密集型、语言/脚本群组、代码模板),这些模式无法通过解码器余弦相似性聚类获得。词元直方图基线整体纯度更高,说明图视图贡献互补而非主导:它揭示了词元频次和解码器权重视角无法捕捉的结构性关系。聚类结果在不同图构建超参数和随机种子下保持稳定。

原文摘要 · Abstract (English)

Sparse autoencoders (SAEs) have become central to mechanistic interpretability, decomposing transformer activations into monosemantic features. Yet existing analyses characterise features almost exclusively through top-activating token lists or decoder weight vectors, leaving the higher-order co-occurrence structure shared across features largely unexamined. We introduce a graph-structured representation in which each SAE feature is modelled as a token co-occurrence graph: nodes are the tokens most frequent near strong activations, and edges connect pairs that co-occur within local context windows. A custom WL-style, frequency-binned graph kernel then provides a similarity measure over this structural space. Applied as a proof of concept to features from a large SAE trained on GPT-2 Small and probed with a synthetic mixed-domain corpus, our clustering recovers heuristic motif families (punctuation-heavy patterns, language and script clusters, and code-like templates) that are not recovered by clustering on decoder cosine similarity. A token-histogram baseline achieves higher overall purity, so the contribution of the graph view is complementary rather than dominant: it surfaces structural relationships that token-frequency and decoder-weight views alone do not capture. Cluster assignments are stable across graph-construction hyperparameters and random seeds.

可解释性图神经网络稀疏自编码器特征分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。