发现CNN学习过程中形状/纹理偏差与测试误差双下降同步，揭示了双下降的内在机制。
On the Relationship Between Double Descent of CNNs and Shape/Texture Bias Under Learning Process
- 通过量化形状与纹理偏差，研究其随训练轮次的变化规律。
- 测试误差双下降阶段，形状/纹理偏差也呈现双下降或双上升趋势。
- 即使无标签噪声，偏差变化仍可出现双下降，说明其机制更复杂。
双下降现象打破了传统偏差-方差权衡理论，但其成因尚未完全明确。在卷积神经网络（CNN）图像识别研究中，已有方法可量化模型对形状与纹理特征的偏好，从而判断其关注重点。本文假设CNN学习过程中的形状/纹理偏差与轮次级双下降存在关联，并进行了验证。结果发现，在出现轮次级双下降的条件下，测试误差的双下降伴随形状/纹理偏差的双下降或双上升。定量分析表明，测试误差与偏差值从初始下降到完全上升阶段具有强相关性。有趣的是，即便在无标签噪声的情况下，部分模型仍观察到形状/纹理偏差的双下降，而此前认为该现象仅出现在含噪声场景。这些发现有助于理解双下降现象及CNN图像识别的学习机制。
原文摘要 · Abstract (English)
The double descent phenomenon, which deviates from the traditional bias-variance trade-off theory, attracts considerable research attention; however, the mechanism of its occurrence is not fully understood. On the other hand, in the study of convolutional neural networks (CNNs) for image recognition, methods are proposed to quantify the bias on shape features versus texture features in images, determining which features the CNN focuses on more. In this work, we hypothesize that there is a relationship between the shape/texture bias in the learning process of CNNs and epoch-wise double descent, and we conduct verification. As a result, we discover double descent/ascent of shape/texture bias synchronized with double descent of test error under conditions where epoch-wise double descent is observed. Quantitative evaluations confirm this correlation between the test errors and the bias values from the initial decrease to the full increase in test error. Interestingly, double descent/ascent of shape/texture bias is observed in some cases even in conditions without label noise, where double descent is thought not to occur. These experimental results are considered to contribute to the understanding of the mechanisms behind the double descent phenomenon and the learning process of CNNs in image recognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。