新优化器GENIE通过动态平衡参数贡献，提升跨域泛化能力。
One-Step Generalization Ratio Guided Optimization for Domain Generalization
- 基于单步泛化比OSGR量化参数贡献，动态调整梯度方向。
- 在多个数据集上显著优于现有优化器，提升跨域准确率。
- 适合需要强泛化性能的模型训练，尤其适用于领域泛化任务。
领域泛化（DG）旨在训练能在未见目标领域表现良好的模型，但常因过度拟合领域特定特征而产生不良相关性。基于梯度的DG方法通常引导梯度沿主导方向，却可能无意中强化虚假相关性。近期工作采用丢弃法正则化过度自信的参数，但未显式调整梯度对齐或确保参数更新的均衡性。本文提出GENIE（Generalization-ENhancing Iterative Equalizer），一种新型优化器，利用单步泛化比（OSGR）量化每个参数对损失减少的贡献并评估梯度对齐。通过预处理因子动态均衡OSGR，GENIE防止少数参数主导优化过程，从而促进领域不变特征学习。理论上，GENIE在参数间平衡收敛贡献与梯度对齐，实现更高OSGR的同时保持SGD的收敛速度。实验表明，其在多个基准测试中超越现有优化器，并可有效提升多种DG及单领域泛化方法的性能。
原文摘要 · Abstract (English)
Domain Generalization (DG) aims to train models that generalize to unseen target domains but often overfit to domain-specific features, known as undesired correlations. Gradient-based DG methods typically guide gradients in a dominant direction but often inadvertently reinforce spurious correlations. Recent work has employed dropout to regularize overconfident parameters, but has not explicitly adjusted gradient alignment or ensured balanced parameter updates. We propose GENIE (Generalization-ENhancing Iterative Equalizer), a novel optimizer that leverages the One-Step Generalization Ratio (OSGR) to quantify each parameter's contribution to loss reduction and assess gradient alignment. By dynamically equalizing OSGR via a preconditioning factor, GENIE prevents a small subset of parameters from dominating optimization, thereby promoting domain-invariant feature learning. Theoretically, GENIE balances convergence contribution and gradient alignment among parameters, achieving higher OSGR while retaining SGD's convergence rate. Empirically, it outperforms existing optimizers and enhances performance when integrated with various DG and single-DG methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。