通过参数敏感性分析,让模型更关注不变特征,提升跨域泛化能力。
DSP-Reg: Domain-Sensitive Parameter Regularization for Robust Domain Generalization
- 基于参数梯度协方差分析,识别对域变化敏感的参数。
- 提出软正则化机制,强化不变参数,抑制域特定参数。
- 在PACS等数据集上平均准确率达66.7%,优于现有方法。
领域泛化(DG)旨在使模型在未见分布的数据上仍保持良好性能,对真实应用至关重要。现有方法多聚焦于学习域不变特征,但忽视了参数层面的深层分析,导致模型难以显式区分对域偏移敏感或鲁棒的参数,影响整体泛化能力。为此,本文构建基于协方差的参数敏感性分析框架,通过计算多个源域间参数梯度的协方差,量化各参数对域变化的敏感度,为方法提供理论基础。在此基础上,提出领域敏感参数正则化(DSP-Reg),一种基于软正则化的优化引导框架,鼓励模型更多依赖域不变参数,抑制域特定参数。该方法实现对学习过程的细粒度控制,在PACS、VLCS、OfficeHome和DomainNet等基准上广泛验证,平均准确率达到66.7%,显著超越所有基线方法。
原文摘要 · Abstract (English)
Domain Generalization (DG) is a critical area that focuses on developing models capable of performing well on data from unseen distributions, which is essential for real-world applications. Existing approaches primarily concentrate on learning domain-invariant features, which assume that a model robust to variations in the source domains will generalize well to unseen target domains. However, these approaches neglect a deeper analysis at the parameter level, which makes the model hard to explicitly differentiate between parameters sensitive to domain shifts and those robust, potentially hindering its overall ability to generalize. In order to address these limitations, we first build a covariance-based parameter sensitivity analysis framework to quantify the sensitivity of each parameter in a model to domain shifts. By computing the covariance of parameter gradients across multiple source domains, we can identify parameters that are more susceptible to domain variations, which serves as our theoretical foundation. Based on this, we propose Domain-Sensitive Parameter Regularization (DSP-Reg), a principled framework that guides model optimization by a soft regularization technique that encourages the model to rely more on domain-invariant parameters while suppressing those that are domain-specific. This approach provides a more granular control over the model's learning process, leading to improved robustness and generalization to unseen domains. Extensive experiments on benchmarks, such as PACS, VLCS, OfficeHome, and DomainNet, demonstrate that DSP-Reg outperforms state-of-the-art approaches, achieving an average accuracy of 66.7\% and surpassing all baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。