CutMix比Cutout更擅长学习稀有特征，提升模型泛化能力。
Provable Benefit of Cutout and CutMix for Feature Learning
- 通过理论分析对比三种训练方式在特征学习上的差异。
- CutMix可学习到比Cutout更稀有的特征，测试准确率最高。
- 适合关注数据增强机制原理的机器学习研究者。
Patch-level 数据增强方法如 Cutout 和 CutMix 在视觉任务中表现出显著效果，但其理论机制仍不清晰。本文研究了三类训练方式：无增强的原始训练、Cutout 训练和 CutMix 训练，针对两层神经网络与特征-噪声数据模型（包含不同稀有度的标签相关特征和不同强度的标签无关噪声）进行分析。理论证明：原始训练无法学习低频特征，Cutout 可以学习低频特征，而 CutMix 能进一步捕捉更稀有的特征。因此，CutMix 的测试准确率最高。我们的分析表明，CutMix 使网络对所有特征和噪声向量的学习趋于均衡，无论其稀有程度或强度如何，为理解 patch-level 增强提供了新视角。
原文摘要 · Abstract (English)
Patch-level data augmentation techniques such as Cutout and CutMix have demonstrated significant efficacy in enhancing the performance of vision tasks. However, a comprehensive theoretical understanding of these methods remains elusive. In this paper, we study two-layer neural networks trained using three distinct methods: vanilla training without augmentation, Cutout training, and CutMix training. Our analysis focuses on a feature-noise data model, which consists of several label-dependent features of varying rarity and label-independent noises of differing strengths. Our theorems demonstrate that Cutout training can learn low-frequency features that vanilla training cannot, while CutMix training can learn even rarer features that Cutout cannot capture. From this, we establish that CutMix yields the highest test accuracy among the three. Our novel analysis reveals that CutMix training makes the network learn all features and noise vectors "evenly" regardless of the rarity and strength, which provides an interesting insight into understanding patch-level augmentation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。