改进凝聚型视觉模型训练，提升性能与效率
RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models
- 采用多分辨率训练与拼贴增强，缓解分辨率偏差和教师不平衡问题
- 提出令牌压缩技术，在固定令牌数下保留高分辨率信息
- 发布多个规模版本，适合高效部署与下游任务迁移
凝聚型模型近期成为训练视觉基础模型的有力方法，通过从CLIP、DINO和SAM等现有模型进行多教师知识蒸馏，实现高效构建鲁棒模型，显著降低计算与资源消耗。本文深入分析当前最先进凝聚型模型，识别出分辨率模式偏移、教师不平衡、教师特有伪影及输出令牌过多等关键挑战。为此，提出多项新方案：多分辨率训练、拼贴增强以及改进教师损失函数平衡。特别在视觉语言模型中，引入令牌压缩技术，在固定令牌数量下保持高分辨率信息。发布多个规模变体（-B, -L, -H, -g），并提供推理代码与预训练权重。
原文摘要 · Abstract (English)
Agglomerative models have recently emerged as a powerful approach to training vision foundation models, leveraging multi-teacher distillation from existing models such as CLIP, DINO, and SAM. This strategy enables the efficient creation of robust models, combining the strengths of individual teachers while significantly reducing computational and resource demands. In this paper, we thoroughly analyze state-of-the-art agglomerative models, identifying critical challenges including resolution mode shifts, teacher imbalance, idiosyncratic teacher artifacts, and an excessive number of output tokens. To address these issues, we propose several novel solutions: multi-resolution training, mosaic augmentation, and improved balancing of teacher loss functions. Specifically, in the context of Vision Language Models, we introduce a token compression technique to maintain high-resolution information within a fixed token count. We release our top-performing variants at multiple scales (-B, -L, -H, and -g), along with inference code and pretrained weights
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。