TRACE可实时分析语言模型训练中的语法、语义发展过程，发现传统指标忽略的深层现象。
TRACE: Training and Inference-Time Interpretability Analysis for Language Models
- 在训练中轻量级追踪语法、语义等表征信号，支持多维度分析。
- 揭示语法先出现、语义后显现、表征压缩等发育现象，超越损失/准确率指标。
- 适合研究者分析模型学习路径，也适用于调试与早期停止策略。
理解语言模型训练过程中语言知识何时及如何涌现，仍是可解释性领域的重要挑战。现有工具多为事后分析，依赖标量指标或需复杂集成，难以部署与维护。我们提出TRACE，一个用于Transformer模型训练与推理阶段可解释性分析的模块化工具包。它支持轻量级的训练中分析，涵盖特征探测、内在维度、海森曲率与输出诊断等信号。其与ABSynth（可控合成语料生成器）集成，提供结构化标注，实现对语言特征习得的精确评估。基于自回归Transformer的实验表明，TRACE揭示了早期句法涌现、延迟语义获取及表征压缩等发展现象，这些信号被传统标量指标（如损失或准确率）所忽视。通过极低集成成本，该工具支持逐层诊断、基于收敛性的早停以及结构错误检测，使Transformer分析更可解释、可操作且可复现。
原文摘要 · Abstract (English)
Understanding when and how linguistic knowledge emerges during language model training remains a central challenge for interpretability. Most existing tools are post hoc, rely on scalar metrics, or require nontrivial integration effort, making comprehensive interpretability analysis difficult to deploy and maintain. We introduce TRACE, a modular toolkit for training and inference-time interpretability analysis of transformer models. It enables lightweight, in-training analysis of linguistic and representational signals, including features probing, intrinsic dimensionality, Hessian curvature, and output diagnostics. It integrates with ABSynth, a controllable synthetic corpus generator that provides structured annotations for precise evaluation of linguistic feature acquisition. Experiments with autoregressive transformers demonstrate that TRACE reveals developmental phenomena such as early syntactic emergence, delayed semantic acquisition, and representational compression, signals overlooked by traditional scalar metrics such as loss or accuracy. With minimal integration effort, the tool enables layer-wise diagnostics, convergence-based early stopping, and detection of structural errors, making transformer analysis interpretable, actionable, and reproducible.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。