发现视觉模型中主干与优化器存在耦合偏差，影响训练效果。
Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning
- 分析主干网络与优化器的相互依赖关系，揭示耦合偏差现象。
- CNNs适配SGD，ViTs等新架构更倾向自适应优化器，影响性能表现。
- 为模型设计提供实证建议，适合关注模型鲁棒性的研究者。
本文深入探讨视觉主干网络与优化器之间的相互作用，揭示了一种称为骨干-优化器耦合偏差（BOCB）的相互依赖现象。我们观察到，经典的CNN如VGG和ResNet与SGD系列优化器表现出显著的协同依赖性，而ViT、ConvNeXt等新型架构则与自适应学习率优化器紧密耦合。进一步研究表明，该偏差既可由优化器引入，也可由特定的主干结构导致，可能显著影响视觉模型的预训练及下游微调表现。通过深入的实证分析，我们总结了推荐优化器选择和稳健主干架构设计的实用建议。希望本工作能引发社区对主干与优化器长期假设的反思，激发更多探索，推动更鲁棒视觉系统的发展。代码与模型已公开于 https://bocb-ai.github.io/。
原文摘要 · Abstract (English)
This paper delves into the interplay between vision backbones and optimizers, unvealing an inter-dependent phenomenon termed \textit{\textbf{b}ackbone-\textbf{o}ptimizer \textbf{c}oupling \textbf{b}ias} (BOCB). We observe that canonical CNNs, such as VGG and ResNet, exhibit a marked co-dependency with SGD families, while recent architectures like ViTs and ConvNeXt share a tight coupling with the adaptive learning rate ones. We further show that BOCB can be introduced by both optimizers and certain backbone designs and may significantly impact the pre-training and downstream fine-tuning of vision models. Through in-depth empirical analysis, we summarize takeaways on recommended optimizers and insights into robust vision backbone architectures. We hope this work can inspire the community to question long-held assumptions on backbones and optimizers, stimulate further explorations, and thereby contribute to more robust vision systems. The source code and models are publicly available at https://bocb-ai.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。