用最大生成树分析语言模型的内部结构，揭示分层表征组织规律。
StructLens: A Structural Lens for Language Models via Maximum Spanning Trees
- 基于残差流语义表示构建最大生成树，从整体结构视角分析模型
- 中间层对连续词元的局部关联组织最强，且训练早期可识别小单元
- 适合研究模型表征演化与结构化学习机制的学者
语言具有内在结构，这一特性解释了语言习得与演变。我们预期语言模型也应具备内部结构。尽管可解释性研究已通过注意力模式和稀疏自编码器分析模型的计算机制，但对表征组织方式仍缺乏关注。为此，我们提出StructLens框架，通过最大生成树分析残差流中的语义表示，受依存句法树启发，提供表征空间中词元关系的总结。结果表明，相邻词元在表征空间中也趋于邻近，中间层表现出最强的局部-跨度组织。对预训练检查点的分析显示，较小的局部单元在训练早期即可被检测到，较大单元则在后期显现。这些发现表明，StructLens能揭示模型在不同层与训练阶段如何组织词元表征。代码已开源：https://github.com/naist-nlp/structlens。
原文摘要 · Abstract (English)
Language exhibits inherent structures, a property that explains both language acquisition and language change. Given this characteristic, we expect language models to manifest their own internal structures as well. While interpretability research has investigated how models compute representations mechanistically through attention patterns and Sparse AutoEncoders, the organization of the resulting representations is overlooked. To address this gap, we introduce StructLens, a framework to analyze representations through a holistic structural view. StructLens constructs maximum spanning trees based on the semantic representations in residual streams, inspired by tree representation in dependency parsing, and provides summaries of token relationships in representation space. We analyze how contiguous tokens are also nearby in representation space and find that middle layers show the strongest local-span organization. Moreover, analysis of pre-training checkpoints reveals that smaller local units become detectable earlier in pre-training, and larger units later. Our findings demonstrate that StructLens provides insights into how models organize token representations across layers and training. Our code is available at https://github.com/naist-nlp/structlens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。