通过频率课程学习加速DINOv2训练并提升抗干扰能力
FastDINOv2: Frequency Based Curriculum Learning Improves Robustness and Training Speed
- 先看低频信息，逐步引入高频内容，配合高斯噪声增强
- 训练时间减少1.6倍，计算量降低2.25倍，鲁棒性不降反升
- 适合追求高效训练和强鲁棒性的自监督视觉模型研究者
大型视觉基础模型如DINOv2凭借庞大架构和海量数据实现优异性能，但在私有数据、新模态或科学探究场景中复现预训练方案仍需极高算力。为此，我们提出一种新型DINOv2预训练策略，可同时加速收敛并提升对常见图像退化的鲁棒性。方法基于频率过滤课程：先处理低频信息，再逐步引入高频内容，并结合高斯噪声补丁增强。在使用ViT-B/16骨干网络于ImageNet-1K上训练时，预训练时间与浮点运算量分别减少1.6倍和2.25倍，且在ImageNet-C基准上保持相当的鲁棒性，线性探测性能也优于基线。该效率与鲁棒性双重提升使大规模自监督建模更易实现，并为数据课程设计与增强方法改善模型鲁棒性提供新路径。代码已开源。
原文摘要 · Abstract (English)
Large-scale vision foundation models such as DINOv2 boast impressive performances by leveraging massive architectures and training datasets. But numerous scenarios require practitioners to reproduce those pre-training solutions, such as on private data, new modalities, or simply for scientific questioning--which is currently extremely demanding computation-wise. We thus propose a novel pre-training strategy for DINOv2 that simultaneously accelerates convergence--and strengthens robustness to common corruptions as a by-product. Our approach involves a frequency filtering curriculum--low-frequency being seen first--and the Gaussian noise patching augmentation. Applied to a ViT-B/16 backbone trained on ImageNet-1K, while pre-training time and FLOPs are reduced by 1.6x and 2.25x, our method still achieves matching robustness in corruption benchmarks (ImageNet-C) and maintains competitive linear probing performance compared with baseline. This dual benefit of efficiency and robustness makes large-scale self-supervised foundation modeling more attainable, while opening the door to novel exploration around data curriculum and augmentation as means to improve self-supervised learning models robustness. The code is available at https://github.com/KevinZ0217/fast_dinov2
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。