选对教师模型，小模型也能大幅提效
How to Choose Your Teacher for Fine Grained Image Recognition
- 根据教师预测比例设计新选择指标
- 比旧方法提升18%选择准确率，学生模型最高增17%精度
- 适合资源受限场景的细粒度图像识别应用
细粒度图像识别用于区分鸟类种类或汽车型号等子类别。尽管当前最先进模型精度高，但往往资源消耗大，难以部署在设备受限场景。知识蒸馏通过将大型教师模型的知识迁移至小型学生模型来缓解此问题。然而，教师模型的选择直接影响学生性能，是关键挑战。本文提出一种基于教师预测比例的新选择指标——Ratio 1-2。在3个学生模型、8个教师模型、8个数据集及4种训练策略下，开展超千次实验验证。结果表明，该指标使教师选择准确率相较此前方法提升18%，助力小模型实现最高达17%的精度提升。实验代码已开源：https://github.com/arkel23/FGIR-KD-Teacher。
原文摘要 · Abstract (English)
Fine-grained image recognition classifies subcategories such as bird species or car models. While state-of-the-art (SOTA) models are accurate, they are often too resource-intensive for deployment on constrained devices. Knowledge distillation addresses this by transferring knowledge from a large teacher model to a smaller student model. A key challenge is selecting the right teacher, as it heavily impacts student performance. This paper introduces a teacher selection metric, \textbf{Ratio 1-2}, based on teacher prediction ratios. Extensive analysis of over one thousand experiments across 3 students, 8 teachers, and 8 datasets under 4 training strategies demonstrates that our metric improves teacher selection by 18\% over previous methods, enabling small student models to achieve up to 17\% accuracy gains. Experiment codebase is available at: \href{https://github.com/arkel23/FGIR-KD-Teacher}{https://github.com/arkel23/FGIR-KD-Teacher}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。