arXiv:2607.18306cs.LGcs.AI2026-07

动态分配扰动预算,让模型更抗干扰、泛化更强

Gradient-Energy Guided Block-Wise Perturbations for Sharpness-Aware Minimization

  • 用梯度能量均值动态感知各参数块的敏感度
  • 在多个任务上提升分类准确率与鲁棒性
  • 无需计算海森向量积,适合各类网络架构

Sharpness-Aware Minimization (SAM) 通过最小化局部参数邻域内的最坏损失来提升泛化能力。标准 SAM 将全局扰动预算按当前小批量梯度范数分布到各参数块,这种分配方式存在噪声且无法反映训练过程中块的累积敏感度。本文提出 Gradient-Energy Adaptive Radius SAM (GEAR-SAM),通过平方块梯度的指数移动平均(EMA)构建轻量级曲率相关敏感度信号,并通过闭式约束优化重新分配固定扰动预算。GEAR-SAM 保持 SAM 的全局半径,无需海森向量积或显式 Fisher 估计,仅额外引入标量状态。在图像分类、迁移学习、噪声标签学习及分块研究中,实验表明其在多种架构和任务上均提升泛化性能与鲁棒性。更广泛地,GEAR-SAM 提供了尖锐性感知优化的动态视角:固定扰动预算应随训练中功能模块敏感度的变化而动态重分配。

原文摘要 · Abstract (English)

Sharpness-Aware Minimization (SAM) improves generalization by minimizing the worst-case loss in a local parameter neighborhood. Standard SAM implicitly allocates its global perturbation budget across parameter blocks according to instantaneous minibatch gradient norms. Such an allocation can be noisy and may not reflect the sensitivity that blocks accumulate throughout training. We propose Gradient-Energy Adaptive Radius SAM (GEAR-SAM), which maintains an exponential moving average (EMA) of squared block gradients as a lightweight, curvature-related sensitivity signal and allocates the fixed SAM budget through a closed-form constrained optimization. GEAR-SAM preserves the global SAM radius, requires no Hessian-vector products or explicit Fisher estimation, and adds only scalar state beyond SAM. Experiments on image classification, transfer learning, noisy-label learning, and partition studies demonstrate improved generalization and robustness across architectures and tasks. More broadly, GEAR-SAM provides a dynamic view of sharpness-aware optimization: a fixed perturbation budget should be redistributed as the sensitivity of functional network blocks evolves during training.

优化算法泛化提升SAM动态分配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。