arXiv:2608.12655cs.LGstat.ML2026-08

提出挑战式训练框架,可检测神经网络是否达到全局最优。

Training Under Challenge: Executable Certificates and Challenge-Closed Optimality for Neural Networks

论文配图:Training Under Challenge: Executable Certificates and Challenge-Closed Optimality for Neural Networks
图 1 · 摘自论文原文
  • 用可执行证书机制构造替代方案并重新评估目标
  • 实测显示误差边界在真实差距的1.74至3.02倍内
  • 适合需要验证模型优化程度的研究者使用

平坦的训练曲线无法揭示神经网络是否达到全局最优、局部陷落、表达受限或与训练器不匹配。我们提出挑战式训练(Training Under Challenge),一种可执行证书框架:预先声明的、符合架构的程序可构建同类别完整替代方案,并重新评估相同目标。任何值更低的候选方案都是可重现的证据,用于下界估计检查点的实证全局最优差距。通过有限套挑战仅能得出相对结论;全局差距判断需额外证明覆盖能力。我们定义资源索引的挑战强度模量,刻画通过挑战所能容忍的最大差距。对于平方损失,当前块下降算子使覆盖性可验证,并给出统一与实现残差界。我们证明反向前沿:无覆盖时,一阶ReLU训练器可达无限多个精确条件头最优解,却收敛于非全局点。在已知最优解的通道门控ResNet-18蒸馏问题中,八个内部挑战覆盖全部240个审计输出方向,实现残差界为真实差距的1.74–3.02倍。成对预测证书可区分解码器使用不足与表达能力不足,量化去噪研究则实现诊断、修复与当前状态再认证。

原文摘要 · Abstract (English)

A flat training curve does not reveal whether a neural network has reached a global optimum, is locally trapped, is representation-limited, or is mismatched to its trainer. We introduce Training Under Challenge, an executable-certificate framework in which predeclared, architecture-valid procedures construct complete alternatives in the same certified class and reevaluate the same objective. Any lower-valued candidate is a replayable witness that lower-bounds the checkpoint's empirical global-optimality gap. Passing a finite suite is only suite-relative; global-gap conclusions require a separately justified coverage mechanism. We define a resource-indexed challenge-power modulus that characterizes the largest gap compatible with passage. For squared loss, current block-decrease operators make coverage checkable and yield uniform and realized-residual bounds. We prove the converse frontier: without coverage, a first-order ReLU trainer can reach infinitely many exact conditional head optima while converging to a non-global point. On a channel-gated ResNet-18 distillation problem with known optimum, eight internal challenges cover all 240 audited output directions, and realized-residual bounds lie within factors of 1.74--3.02 of the true gap. Paired predictive certificates separate decoder under-use from representation insufficiency, while quantized-denoising studies demonstrate diagnosis, repair, and current-state recertification.

神经网络优化可验证训练模型诊断

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。