通过参考轨迹检测低精度训练中的隐性权重更新，提升模型可解释性。
Auditing Invisible Weight Updates with Reference Traces
- 用高精度参考轨迹追踪低精度训练中的权重变化过程。
- 55/72 网格单元的预测与实测交叉时间匹配度达94.5%，误差在15%内。
- 适用于模型审计、微调分析及对训练稳定性敏感的研究者。
直接低精度写回会抹除非零优化器提议。我们探究在低精度运行前，高精度参考轨迹能提供什么信息。目标代码事件可在实际目标轨迹上逐坐标审计；预运行的聚合投影也假设参考轨迹仍具参考价值。在受控两层网格中，55/72个单元实现了测量与预测的初始化后交叉：时间跨度达384倍，52/55在15%误差内，4/72分类不一致。匹配解码器实验表明，随机而非最近写回能恢复大部分损失差距。前瞻性分析网格E4M3审计复用一个fp32轨迹，覆盖三个未见的NeoX风格种子。其通过绝对精度与技能门限（宏均方根误差0.00858），但方向特异性失败。在目标结果不可知比较中，历史模板描述均方根误差（0.00360）低于预声明源预测器（0.00438）；事后分解显示99.65%变异由共同时期主导，而特权匹配参考校正可达0.00283。持久原生研究1在两个调度间配对三个种子，五单元为典型；第六个手动单元缺乏典型过程标识，故结果仍不确定。回顾性协议偏差分析为阴性，因完整常中组与恢复的余弦重启单元无交集。研究2报告全-子域/死区子域恢复率分别为0.9766/0.9777，比例1.0012，体现策略差异而非因果中介。模拟-INT3研究3重播六个检查点，观察到单固定种子验证损失降低7.3071纳特（69.71%）。精确事件与写回效应可审计，但聚合预测可能反映共享时间而非源特定迁移。
原文摘要 · Abstract (English)
Direct low-precision write-back can erase nonzero optimizer proposals. We ask what a high-precision reference trace establishes before a low-precision run. The exact target-code event is auditable coordinatewise on a realized target trajectory; pre-run aggregate projection also assumes the reference remains a useful counterfactual. In a controlled two-layer grid, 55/72 cells have measured and predicted post-initialization crossings: times span $384\times$, 52/55 are within 15\%, and 4/72 differ in category. Matched decoder experiments show stochastic rather than nearest write-back recovers most of the loss gap. A prospective analytic-grid E4M3 audit reuses one fp32 trace across three unseen NeoX-style seeds. It passes absolute-accuracy and skill gates (macro RMSE 0.00858) but fails directional specificity. In a target-outcome-blind comparison, a historical template has lower descriptive RMSE (0.00360) than the predeclared source predictor (0.00438); a post-outcome decomposition assigns 99.65\% of variation to common time, while a privileged matched-reference correction reaches 0.00283. Persistent-native Study~1 pairs three seeds across two schedules. Five cells are canonical; a manual sixth lacks canonical process identity, so the registered result remains inconclusive. A retrospective protocol-deviation analysis is negative because the complete constant-mid cohort is disjoint from the recovered cosine-restart cell. Study~2 reports mean full-SR/dead-zone-SR recoveries of 0.9766/0.9777 and a ratio of 1.0012, a policy contrast rather than causal mediation. Simulated-INT3 Study~3 replays six checkpoints and observes a 7.3071-nat (69.71\%) validation-loss reduction in one fixed seed. Exact events and write-back effects are auditable, but aggregate forecasts can reflect shared time rather than source-specific transfer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。