arXiv:2607.21999cs.LGcs.AI2026-07

通过动态调整采样策略,让模型更均衡地探索平坦的损失区域。

From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning

  • 基于类别的频率与梯度尖锐度反馈,动态调节采样概率
  • 在CIFAR-100 LT上尾部准确率提升10.85点,整体提升3.56点
  • 无需额外反向传播,训练时间仅是标准SAM的1.02倍

长尾学习面临两个泛化瓶颈:头部类别过度暴露,而尾部类别常收敛至损失曲面的尖锐区域。传统重采样仅缓解暴露偏差,现有尖锐度感知最小化(SAM)方法在已有偏差小批量后才调整损失或扰动。本文提出尖锐度引导平衡采样(SGS),将采样分布作为优化几何的主动控制变量。SGS通过累积类别频次和标准SAM更新获得的指数移动平均(EMA)尖锐度估计,动态增加低频类别的采样概率,并抑制因SAM产生大损失变化的类别,无需类别级扰动或额外反向传播。我们通过连续时间随机微分方程与采样依赖的PAC-Bayes分析刻画该过程,揭示频率-尖锐度反馈如何推动训练趋向更均衡的平坦性分布。在不平衡比为100的CIFAR-100 LT上,SGS-SAM相比Focal-SAM尾部准确率提升10.85点,整体提升3.56点;在ImageNet-LT上,相比ImbSAM尾部提升6.59点,整体提升1.20点。训练时间仅为标准SAM的1.02倍。此方法开辟了采样侧控制损失曲面的新路径,表明未来长尾方法可协同调控数据暴露与优化几何,而非固定其一。

原文摘要 · Abstract (English)

Long-tailed learning couples two sources of poor generalization: head classes dominate training exposure, while under-represented classes often converge to sharper regions of the loss landscape. Conventional re-sampling addresses the former without considering geometry, whereas existing long-tailed sharpness-aware minimization (SAM) methods modify losses or perturbations only after biased mini-batches have been drawn. We introduce Sharpness-Guided Equilibrium Sampling (SGS), which treats the sampling distribution as an active control variable for optimization geometry. SGS dynamically adjusts subsequent mini-batches by increasing the sampling probability of less frequently sampled classes while suppressing classes with large SAM-induced loss changes, using only cumulative class counts and EMA sharpness estimates obtained from the standard SAM update, without class-wise perturbations or additional backward passes. We characterize this sampling process through a continuous-time stochastic differential equation and a sampling-dependent PAC-Bayes analysis, explaining how frequency-sharpness feedback can move training toward a more balanced flatness profile. On CIFAR-100 LT with an imbalance ratio of 100, SGS-SAM improves Focal-SAM by 10.85 points in tail accuracy and 3.56 points overall. On ImageNet-LT, it improves ImbSAM by 6.59 points on tail classes and 1.20 points overall. Its training time is only $1.02\times$ that of vanilla SAM. Beyond these gains, SGS establishes a sampling-side route to loss-landscape control, suggesting that future long-tailed methods can jointly regulate data exposure and optimization geometry rather than treating either as fixed.

长尾学习采样优化尖锐度感知模型泛化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。