为神经网络训练设计自动稳定与恢复机制，提升训练鲁棒性。
Automatic Stability and Recovery for Neural Network Training
- 通过验证探针提取创新信号，实时检测不稳定性
- 实现无需修改优化器的自动恢复，保障训练过程安全
- 适用于内存受限场景，开销极小，适合工业级应用
现代神经网络训练日益脆弱，罕见但严重的不稳定更新常导致不可逆发散或隐性性能下降。现有优化方法主要依赖优化器内置的预防机制，对已发生不稳定缺乏检测与恢复能力。本文提出一种监督式运行时稳定性框架，将优化视为受控随机过程。通过分离来自验证探针等次级测量的创新信号，该框架可在不修改底层优化器的前提下，自动检测并恢复不稳定更新。我们提供了理论上的运行时安全保证，形式化定义了退化范围与恢复能力。实现开销极低，兼容内存受限的训练环境。
原文摘要 · Abstract (English)
Training modern neural networks is increasingly fragile, with rare but severe destabilizing updates often causing irreversible divergence or silent performance degradation. Existing optimization methods primarily rely on preventive mechanisms embedded within the optimizer, offering limited ability to detect and recover from instability once it occurs. We introduce a supervisory runtime stability framework that treats optimization as a controlled stochastic process. By isolating an innovation signal derived from secondary measurements, such as validation probes, the framework enables automatic detection and recovery from destabilizing updates without modifying the underlying optimizer. We provide theoretical runtime safety guarantees that formalize bounded degradation and recovery. Our implementation incurs minimal overhead and is compatible with memory-constrained training settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。