语言模型的特征有生命周期,关键特征早期形成并主导模型结构。
Features have life history. And we should care
- 发现模型中存在约50个稳定特征,构成早期形成的表征骨架。
- 骨架特征在训练初期快速形成,且能承载99%训练后期的表征发展。
- 仅凭训练初期的激活模式就能预判哪些特征会成为骨干,准确率达80%。
语言模型中的特征具有生命周期:它们在训练过程中出现、持续存在并最终消亡,但这一过程的重要性尚未被充分研究。我们在 Pythia-160M 和 -410M 模型中发现一个持久的表征主干,称为载体骨架(carrier scaffold),包含约50个稀疏特征,其生命周期稳定,并组织起整个模型的表征结构。该骨架具备四个特性:(i) 早期组装:前1%训练阶段特征的涌现、消亡与重组速度比后续快约40倍,骨架在此时已基本固定;(ii) 承载能力强:跨层联合消融实验表明,骨架特征的承载力远超同等数量的非骨架特征,此差异无法通过单特征激活方法捕捉;(iii) 功能先于方向:仅凭训练初始阶段的激活模式,即可在5组中正确预测4组未来成为骨架的特征,早于模型几何结构稳定;(iv) 驱动后续发展:训练结束时,骨架特征已招募64%的活跃特征进入其层级结构。生命史支持两阶段训练理论:前1%主要完成骨架选择,后99%则围绕已定基础进行几何校准。
原文摘要 · Abstract (English)
Features in language models have life history: they emerge, persist, and die during training, yet the importance of that history remains largely unexplored. We find evidence of a persistent representational backbone, which we identify in Pythia-160M and -410M as the carrier scaffold: ${\sim}50$ sparse features with stable life histories, around which the model's representational structure organises. It has four properties. \emph{(i)}~\emph{It assembles early:} features emerge, die, and reorganise ${\sim}40\!\times$ faster in the first $1\%$ of training than afterwards, and the scaffold is already largely fixed by then. \emph{(ii)}~\emph{It is load-bearing:} joint cross-layer ablation identifies the carriers as far more load-bearing than any count-matched non-scaffold population, a gap invisible to per-firing single-feature methods. \emph{(iii)}~\emph{Function precedes direction:} which features will become carriers is already predictable from training-onset firing patterns alone, correctly distinguishing future carriers from non-carriers in $4$ of $5$ cases, before the geometry has settled. \emph{(iv)}~\emph{It seeds subsequent development:} by the end of training, scaffold carriers have recruited $64\%$ of all active features into the scaffold hierarchy. Life history is consistent with a two-phase account of training: selection appears to largely determine the scaffold in the first $1\%$; the remaining $99\%$ appears to calibrate geometry around a substrate already set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。