让学生模型自己学会筛选重要目标，提升知识蒸馏效果。
Learnable Instance Attention Filtering for Adaptive Detector Distillation
- 学生模型动态学习目标重要性，实现自适应筛选。
- 在KITTI和COCO上提升2%，不增加计算开销。
- 适合追求高效检测器部署的研究者和工程师。
随着深度视觉模型复杂度上升以追求更高性能，部署效率成为关键挑战。知识蒸馏（KD）通过将大型教师模型的知识迁移至紧凑的学生模型来缓解此问题。尽管许多基于特征的KD方法依赖空间过滤引导蒸馏，但通常对所有目标实例一视同仁，忽略了实例级别的差异性。此外，现有的注意力过滤机制多为启发式或由教师驱动，而非由学生学习获得。为此，我们提出可学习实例注意力过滤的自适应检测器蒸馏框架（LIAF-KD），引入可学习的实例选择器，在蒸馏过程中动态评估并重新加权实例重要性。值得注意的是，该过程由学生模型根据其演进的学习状态参与完成。在KITTI和COCO数据集上的实验表明，该方法实现一致改进：在无额外复杂度的GFL ResNet-50学生模型上获得2%的提升，优于现有最先进方法。
原文摘要 · Abstract (English)
As deep vision models grow increasingly complex to achieve higher performance, deployment efficiency has become a critical concern. Knowledge distillation (KD) mitigates this issue by transferring knowledge from large teacher models to compact student models. While many feature-based KD methods rely on spatial filtering to guide distillation, they typically treat all object instances uniformly, ignoring instance-level variability. Moreover, existing attention filtering mechanisms are typically heuristic or teacher-driven, rather than learned with the student. To address these limitations, we propose Learnable Instance Attention Filtering for Adaptive Detector Distillation (LIAF-KD), a novel framework that introduces learnable instance selectors to dynamically evaluate and reweight instance importance during distillation. Notably, the student contributes to this process based on its evolving learning state. Experiments on the KITTI and COCO datasets demonstrate consistent improvements, with a 2% gain on a GFL ResNet-50 student without added complexity, outperforming state-of-the-art methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。