用频域解耦注意力提升小模型检测精度，效果优于传统方法。
FD-CanKD: Frequency-Decoupled Cross-Attention Distillation as a Refinement Prior for Compact Object Detectors
- 通过三重知识迁移：预测监督、关系上下文传递、频域组件对齐
- 在COCO上达48.87 mAP50:95，比基线提升显著
- 适合部署在资源受限设备的小型目标检测器优化
紧凑型目标检测器适用于资源受限的视觉感知任务，但其表征能力有限，导致精度低于大型模型。传统检测器蒸馏多依赖预测层监督或单一特征对齐目标，如响应、分布、相关性或频域匹配。本文提出面向检测器的频率解耦交叉注意力知识蒸馏（FD-CanKD）框架，从三个互补层面转移教师知识：头部预测监督、关系级非局部上下文传递、频率级组件选择性对齐。学生特征首先通过基于交叉注意力的关系传递聚合教师侧空间上下文，随后频域感知对齐保留互补的结构与细节敏感线索。在控制条件下，固定50轮从头训练的对比实验显示，FD-CanKD在性能上保持与代表性检测器蒸馏基线相当。蒸馏后继续微调20轮，生成更强的可精炼学生模型，达到48.87 mAP50:95、65.84 mAP50、53.40 mAP75。所有蒸馏模块在训练后移除，部署学生模型参数量为19.7M。该框架在代表性的YOLOv12教师-学生设置中实例化并评估，作为紧凑检测器案例研究。
原文摘要 · Abstract (English)
Compact object detectors are suitable for resource-constrained visual perception, but their limited representation capacity creates an accuracy gap relative to large models. Conventional detector distillation often relies on prediction-level supervision or a single feature-alignment target, such as response, distribution, correlation, or frequency-domain matching. Frequency-Decoupled Cross-Attention Knowledge Distillation (FD-CanKD) is presented as a detector-oriented framework that transfers teacher knowledge at three complementary levels: head-level prediction supervision, relation-level non-local context transfer, and frequency-level component-selective alignment. Student features first aggregate teacher-side spatial context through cross-attention-based relation transfer, after which frequency-aware alignment preserves complementary structural and detail-sensitive cues. Under controlled Microsoft Common Objects in Context (COCO) experiments, fixed 50-epoch from-scratch comparisons show that FD-CanKD remains competitive with representative detector knowledge distillation baselines. Post-distillation continued fine-tuning further produces a stronger refinement-ready student than detector-only fine-tuning, reaching 48.87 mean average precision (mAP) at intersection-over-union thresholds from 0.50 to 0.95 (mAP50:95), 65.84 mAP50, and 53.40 mAP75 after 20 additional epochs. All distillation modules are removed after training, leaving the deployed student unchanged at 19.7M parameters. The framework is instantiated and evaluated in a controlled YOLOv12 teacher-student setting as a representative compact-detector case study.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。