让神经网络像材料一样感知压力,自动调整训练策略以提升泛化能力。
Stress-Aware Resilient Neural Training
- 基于材料疲劳原理,用内部应力信号触发自适应噪声注入。
- 在7个视觉任务上显著提升模型泛化性,计算开销几乎为零。
- 适合追求鲁棒训练的开发者和研究者,尤其关注模型稳定性时。
本文提出应力感知学习(Stress-Aware Learning),一种新型抗干扰神经网络训练范式。受材料科学中弹性与塑性形变启发,该方法通过检测训练过程中的损失与准确率停滞信号(即内部应力)来判断优化困难程度,并动态注入自适应噪声,帮助模型逃离尖锐极小值,收敛至更平坦、更具泛化性的损失曲面区域。所提出的塑性形变优化器(Plastic Deformation Optimizer)在六种网络架构、四种优化器及七个视觉基准测试中验证了其有效性,均实现更强的鲁棒性与泛化性能,且计算开销极低。代码与3D可视化资源将开源于GitHub:https://github.com/Stress-Aware-Learning/SAL。
原文摘要 · Abstract (English)
This paper introduces Stress-Aware Learning, a resilient neural training paradigm in which deep neural networks dynamically adjust their optimization behavior - whether under stable training regimes or in settings with uncertain dynamics - based on the concept of Temporary (Elastic) and Permanent (Plastic) Deformation, inspired by structural fatigue in materials science. To instantiate this concept, we propose Plastic Deformation Optimizer, a stress-aware mechanism that injects adaptive noise into model parameters whenever an internal stress signal - reflecting stagnation in training loss and accuracy - indicates persistent optimization difficulty. This enables the model to escape sharp minima and converge toward flatter, more generalizable regions of the loss landscape. Experiments across six architectures, four optimizers, and seven vision benchmarks demonstrate improved robustness and generalization with minimal computational overhead. The code and 3D visuals will be available on GitHub: https://github.com/Stress-Aware-Learning/SAL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。