针对模型依赖特定频率的捷径学习问题,提出动态调整数据频域结构的新方法。
Towards Combating Frequency Simplicity-biased Learning for Domain Generalization
- 通过操控数据在傅里叶域的统计结构,干预模型对频率特征的学习行为。
- 在多个域泛化基准上提升性能,尤其在未见目标域上效果显著。
- 适合关注模型鲁棒性与避免学习捷径的研究者使用。
领域泛化方法旨在从源域中学习可迁移的知识,以在未见目标域上实现良好泛化。近期研究表明,神经网络常表现出对简单性偏好的学习行为,过度依赖特定频率成分(即频率捷径),而非语义信息,导致泛化能力下降。尽管已有数据增强技术提升了泛化性能,但其往往引入更多频率捷径,造成泛化提升的假象。本文从数据驱动视角出发,旨在防止此类捷径学习。基于模型在不同空间频率成分上的偏差学习行为与数据集频率特性的理论关联,我们提出通过改变数据在傅里叶域的统计结构,来调控模型对各频率成分的学习。直观上,由于频率捷径隐藏于数据结构中的主导且高度相关的频率中,动态扰动这些过依赖的频率成分可抑制捷径应用。为此,我们设计了两个协同自适应的数据增强模块,用于动态调整数据频率特征,从而影响模型学习行为,最终缓解捷径学习。代码已公开于AdvFrequency(https://github.com/C0notSilly/AdvFrequency)。
原文摘要 · Abstract (English)
Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. Recent studies show that neural networks frequently suffer from a simplicity-biased learning behavior which leads to over-reliance on specific frequency sets, namely as frequency shortcuts, instead of semantic information, resulting in poor generalization performance. Despite previous data augmentation techniques successfully enhancing generalization performances, they intend to apply more frequency shortcuts, thereby causing hallucinations of generalization improvement. In this paper, we aim to prevent such learning behavior of applying frequency shortcuts from a data-driven perspective. Given the theoretical justification of models' biased learning behavior on different spatial frequency components, which is based on the dataset frequency properties, we argue that the learning behavior on various frequency components could be manipulated by changing the dataset statistical structure in the Fourier domain. Intuitively, as frequency shortcuts are hidden in the dominant and highly dependent frequencies of dataset structure, dynamically perturbating the over-reliance frequency components could prevent the application of frequency shortcuts. To this end, we propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning. Code is available at AdvFrequency (https://github.com/C0notSilly/AdvFrequency).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。