动态调整正则化参数，让模型更稳定、泛化更好
Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural Networks
- 用可变的尖锐度比率自动调节正则化强度
- 在CIFAR-10和CIFAR-100上提升准确率与泛化能力
- 理论证明收敛性，适合追求高泛化的深度学习研究者
Sharpness-Aware Minimization (SAM) 在提升模型泛化性能方面表现优异。然而，其使用固定超参数来表征模型尖锐度，缺乏自适应性。本文提出 SAM with Adaptive Regularization (SAMAR)，引入灵活的尖锐度比规则，动态更新正则化参数。我们为满足Lipschitz连续性的函数提供了SAMAR的收敛性理论证明。在图像识别任务中，基于CIFAR-10和CIFAR-100的实验表明，SAMAR显著提升了模型准确率与泛化能力。
原文摘要 · Abstract (English)
Sharpness-Aware Minimization (SAM) has proven highly effective in improving model generalization in machine learning tasks. However, SAM employs a fixed hyperparameter associated with the regularization to characterize the sharpness of the model. Despite its success, research on adaptive regularization methods based on SAM remains scarce. In this paper, we propose the SAM with Adaptive Regularization (SAMAR), which introduces a flexible sharpness ratio rule to update the regularization parameter dynamically. We provide theoretical proof of the convergence of SAMAR for functions satisfying the Lipschitz continuity. Additionally, experiments on image recognition tasks using CIFAR-10 and CIFAR-100 demonstrate that SAMAR enhances accuracy and model generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。