arXiv:2606.11375cs.CLcs.AI2026-06被引 3

探针准确率饱和后,用脆弱性指标揭示语言模型预训练中的隐藏结构。

When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis

论文配图:When Probing Accuracy Saturates, Fragility Resolves: A Complementary Metric for LLM Pre-Training Analysis
图 1 · 摘自论文原文
  • 提出脆弱性指标:衡量激活噪声下探针准确率崩溃的临界点。
  • 发现道德词汇编码先出现,组合性表征后形成,准确率无法捕捉此过程。
  • 建议使用归一化噪声做跨层比较,原始噪声用于层内对比。

标准线性探针在隐藏状态上达到高准确率时即认为某属性已被编码。该方法在预训练初期有效,但探针准确率在前几千步即趋于饱和,使后续训练过程不可见。本文引入‘脆弱性’作为互补指标,定义为探针准确率崩溃时的激活噪声水平。脆弱性同时敏感于可分性边际与表征冗余度,二者在准确率平稳后仍持续演化。应用于开源检查点语言模型,脆弱性揭示了准确率无法察觉的结构。我们关注的道德化表征以阶段式演进:道德词首次被高准确率检测,组合性编码随后出现。由于探针准确率仅反映数据集的词法可分性,我们通过展示组合编码能跨无共同对比词的构造类型迁移,直接验证其存在。当准确率呈平坦响应时,脆弱性呈现结构性变化。原始临界噪声亦显示层深梯度,但我们识别并修正了一个此前未被记录的混杂因素:深层激活幅值更大,固定绝对噪声会使深层显得更鲁棒。按层归一化可消除大部分梯度,层内比较依然有效,并能区分相同准确率的数据集。建议对跨层比较使用均方根归一化临界噪声,层内比较则用原始临界噪声。

原文摘要 · Abstract (English)

Standard linear probing declares a property "encoded" when a classifier on hidden states achieves high accuracy. The protocol works well on a snapshot but breaks across pre-training, with probe accuracy saturating within the first few thousand steps, leaving most of training invisible to the instrument. We introduce fragility, a complementary per-layer metric defined as the activation-noise level at which probe accuracy collapses. Fragility is sensitive to both the margin of separability and the redundancy of representation, both of which keep evolving long after accuracy plateaus. Applied to open-checkpoint language models, fragility recovers structure that accuracy alone cannot see. Moralized representations, our interest, emerge in stages, with high-accuracy detection of morally loaded words first and compositional encoding later. Because probe accuracy on its own tracks how lexically separable a dataset is, we establish the compositional encoding directly, by showing it transfers across construction types that share no contrast tokens. Where probing accuracy returns a flat answer, fragility returns a structured one. Raw critical noise also shows a layer-depth gradient, but we identify and correct a confound that, to our knowledge, has gone undocumented in noise-injection probing. That is, deeper layers carry larger activations, so a fixed absolute noise makes them look more robust than they are. Rescaling per layer removes most of the gradient, while within-layer comparisons stay valid and still separate identical-accuracy corpora by fragility. We recommend RMS-normalized critical noise for cross-layer claims and raw critical noise for within-layer comparisons.

大模型分析探针评估表征演化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。