提出DGSAM,通过优化每个源域的尖锐度提升跨域泛化能力。
DGSAM: Domain Generalization via Individual Sharpness-Aware Minimization
- 针对每个源域分别进行尖锐度感知优化,避免虚假平坦最小值
- 在多个数据集上平均准确率提升,域间性能方差显著降低
- 计算开销低于SAM,适合实际部署的域泛化任务
域泛化(DG)旨在通过在多个源域上训练,使模型在未见目标域上表现良好。虽然尖锐度感知最小化(SAM)因能寻找平坦极小值以提升泛化能力而被广泛采用,但我们的分析发现,在DG中SAM可能收敛到‘虚假平坦极小值’:全局尖锐度看似平坦,但各源域内部仍保持高尖锐度。为此,我们形式化了平均最坏域风险,即在有界分布偏移下的最大损失,并推导出泛化界,揭示了全局尖锐度感知最小化的局限性。相反,个体尖锐度可作为该风险的有效上界,更适合作为鲁棒域泛化的代理。基于此,我们提出降低开销的渐进式域感知SAM(DGSAM),以计算高效的方式对各源域施加渐进式扰动,持续减小个体尖锐度。大量实验表明,DGSAM不仅提升平均准确率,还显著降低域间性能方差,且计算开销低于SAM。
原文摘要 · Abstract (English)
Domain generalization (DG) aims to learn models that perform well on unseen target domains by training on multiple source domains. Sharpness-Aware Minimization (SAM), known for finding flat minima that improve generalization, has therefore been widely adopted in DG. However, our analysis reveals that SAM in DG may converge to \textit{fake flat minima}, where the total loss surface appears flat in terms of global sharpness but remains sharp with respect to individual source domains. To understand this phenomenon more precisely, we formalize the average worst-case domain risk as the maximum loss under domain distribution shifts within a bounded divergence, and derive a generalization bound that reveals the limitations of global sharpness-aware minimization. In contrast, we show that individual sharpness provides a valid upper bound on this risk, making it a more suitable proxy for robust domain generalization. Motivated by these insights, we shift the DG paradigm toward minimizing individual sharpness across source domains. We propose \textit{Decreased-overhead Gradual SAM (DGSAM)}, which applies gradual domain-wise perturbations in a computationally efficient manner to consistently reduce individual sharpness. Extensive experiments demonstrate that DGSAM not only improves average accuracy but also reduces performance variance across domains, while incurring less computational overhead than SAM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。