融合DINO与巴洛孪生，用更少标签和算力训练出更强的视觉Transformer。
DinoTwins: Combining DINO and Barlow Twins for Robust, Label-Efficient Vision Transformers
- 结合冗余消除与自蒸馏策略，提升模型鲁棒性。
- 仅用10%标注数据，性能媲美独立DINO模型。
- 适合资源受限环境下的高效视觉模型训练。
在无需昂贵标注数据的情况下训练图像理解模型仍是挑战。本文将两种自监督学习方法——DINO（师生学习）与巴洛孪生（冗余消除）相结合，构建一个在少量标签和低计算资源下表现更优的模型。尽管DINO和巴洛孪生各自表现出色，但存在局限：DINO对特定增强敏感，巴洛孪生常需过大批量，难以在消费级硬件上运行。通过整合巴洛孪生的冗余消除目标与DINO的自蒸馏机制，本研究旨在发挥两者互补优势。在仅使用10%标注数据的MS COCO数据集上进行线性探测训练，结果表明该混合模型在损失和分类准确率上与DINO相当，并保持了优异的特征表示能力。注意力可视化进一步显示其具备更强的语义分割潜力。该方法为资源受限环境下视觉变换器的训练提供了可扩展、标签高效的替代方案。
原文摘要 · Abstract (English)
Training AI models to understand images without costly labeled data remains a challenge. We combine two techniques--DINO (teacher-student learning) and Barlow Twins (redundancy reduction)--to create a model that learns better with fewer labels and less compute. While both DINO and Barlow Twins have independently demonstrated strong performance in self-supervised learning, each comes with limitations--DINO may be sensitive to certain augmentations, and Barlow Twins often requires batch sizes too large to fit on consumer hardware. By combining the redundancy-reduction objective of Barlow Twins with the self-distillation strategy of DINO, we aim to leverage their complementary strengths. We train a hybrid model on the MS COCO dataset using only 10\% of labeled data for linear probing, and evaluate its performance against standalone DINO and Barlow Twins implementations. Preliminary results show that the combined approach achieves comparable loss and classification accuracy to DINO while maintaining strong feature representations. Attention visualizations further suggest improved semantic segmentation capability in the hybrid model. This combined method offers a scalable, label-efficient alternative for training ViTs in resource-constrained environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。