用跨时间编码追踪大模型学语言的全过程，看清关键能力何时出现。
Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining
- 用稀疏交叉编码器对齐不同训练阶段的特征
- 发现不规则复数等语言能力在特定阶段突然涌现
- 适合研究模型内部表示演化的学者与开发者
大语言模型在预训练过程中会习得复杂的抽象能力，例如识别不规则复数名词主语。然而，传统评估方法（如基准测试）无法揭示这些能力是如何逐步形成的。为填补这一空白，本文使用稀疏交叉编码器（sparse crosscoders）来发现并对齐模型检查点间的特征。通过在具有显著性能与表征变化的开源检查点三元组间训练交叉编码器，并引入新指标相对间接效应（RelIE），我们能够追踪每个特征对任务表现产生因果影响的关键训练阶段。结果表明，该方法可有效检测特征的出现、维持与消失。该方法具备架构无关性与可扩展性，为预训练期间表示学习的可解释性与细粒度分析提供了新路径。
原文摘要 · Abstract (English)
Large language models (LLMs) learn non-trivial abstractions during pretraining, such as detecting irregular plural noun subjects. However, because traditional evaluation methods (e.g., benchmarking) fail to reveal how models acquire these concepts and capabilities, it is not well understood when and how these specific linguistic abilities emerge. To bridge this gap and better understand model training at the concept level, we use sparse crosscoders to discover and align features across model checkpoints. Using this approach, we track the evolution of linguistic features during pretraining. We train crosscoders between open-sourced checkpoint triplets with significant performance and representation shifts, and introduce a novel metric, Relative Indirect Effects (RelIE), to trace training stages at which individual features become causally important for task performance. We show that crosscoders can detect feature emergence, maintenance, and discontinuation during pretraining. Our approach is architecture-agnostic and scalable, offering a promising path toward more interpretable and fine-grained analysis of representation learning throughout pretraining.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。