用光谱学思想分析语言模型，发现语义模式的聚类规律。
Towards Spectroscopy: Susceptibility Clusters in Language Models
- 通过扰动输入词元，测量模型响应以构建敏感度矩阵。
- 在Pythia-14M上识别出510个可解释聚类，涵盖语法与数学符号等。
- 结果与稀疏自编码器匹配率达50%，验证了方法有效性。
光谱学通过测量系统对扰动的响应来推断其内部结构。我们将其应用于神经网络：在上下文x中提高词元y的权重，利用随机梯度朗之万动力学（SGLD）在局部吉布斯后验上计算组件可观测量与扰动之间的协方差，得到模型响应的敏感度χ_xy。理论上，敏感度可分解为数据分布模式的叠加，解释了为何语义上“因相似原因”出现的词元会在敏感度空间中聚类。实证上，我们将该方法应用于Pythia-14M，开发了一种基于电导的聚类算法，识别出510个可解释的聚类，涵盖语法模式、代码结构、数学符号等。与稀疏自编码器（SAE）对比，50%的聚类与SAE特征匹配，验证了两种方法均能恢复相同内在结构。
原文摘要 · Abstract (English)
Spectroscopy infers the internal structure of physical systems by measuring their response to perturbations. We apply this principle to neural networks: perturbing the data distribution by upweighting a token $y$ in context $x$, we measure the model's response via susceptibilities $χ_{xy}$, which are covariances between component-level observables and the perturbation computed over a localized Gibbs posterior via stochastic gradient Langevin dynamics (SGLD). Theoretically, we show that susceptibilities decompose as a sum over modes of the data distribution, explaining why tokens that follow their contexts "for similar reasons" cluster together in susceptibility space. Empirically, we apply this methodology to Pythia-14M, developing a conductance-based clustering algorithm that identifies 510 interpretable clusters ranging from grammatical patterns to code structure to mathematical notation. Comparing to sparse autoencoders, 50% of our clusters match SAE features, validating that both methods recover similar structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。