用任务专精蒸馏让小ViT在边缘设备上高效做密集预测。
EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation
- 用检测任务蒸馏训练小ViT，提升特定任务表征能力。
- 小模型仅1000万参数就达51.7盒AP，分割和姿态达48.4与74.8。
- 无需额外标注，适合资源受限的边缘部署场景。
在资源受限的边缘设备上部署高性能密集预测模型仍具挑战，因计算与内存预算严格。目前轻量级目标检测、实例分割和姿态估计系统仍以YOLO等CNN架构为主，而紧凑型视觉变换器（ViTs）即使经大规模预训练，也难达同等精度-效率平衡。我们认为此差距源于小型ViTs缺乏任务特异性表征学习，而非ViTs与边缘密集预测天然不匹配。为此，我们提出EdgeCrafter——一种面向边缘密集预测的统一紧凑ViT框架，核心为基于蒸馏的小型主干网络和边缘友好编码器-解码器结构的ECDet检测模型。该检测蒸馏表示通过轻量级任务专用预测模块直接迁移至实例分割与人体姿态估计。无需使用除COCO外的任务标注，ECDet-S实现少于1000万参数下51.7盒平均精度（box AP），ECInsSeg-X与ECPose-X分别达到48.4掩码平均精度（mask AP）和74.8关键点平均精度（keypoint AP）。作为补充但更耗算力的设置，Objects365检测预训练在所有规模上持续提升性能，其中X系列达59.9盒AP、49.8掩码AP和75.9关键点AP。结果表明，结合任务专精蒸馏与边缘感知设计，紧凑型ViTs可成为边缘密集预测的实用且有竞争力的方案。代码已公开：https://intellindust-ai-lab.github.io/projects/EdgeCrafter
原文摘要 · Abstract (English)
Deploying high-performance dense prediction models on resource-constrained edge devices remains challenging due to strict computation and memory budgets. In practice, lightweight systems for object detection, instance segmentation, and pose estimation are still dominated by CNN-based architectures such as YOLO, while compact Vision Transformers (ViTs) often struggle to achieve comparable accuracy-efficiency trade-offs, even with large-scale pretraining. We argue that this gap arises primarily from insufficient task-specific representation learning in small-scale ViTs, rather than from an inherent mismatch between ViTs and edge dense prediction. To address this issue, we introduce EdgeCrafter, a unified compact ViT framework for edge dense prediction centered on ECDet, a detection model built on a distilled compact backbone and an edge-friendly encoder-decoder design. The resulting detection-distilled representation transfers directly to instance segmentation and human pose estimation through lightweight task-specific prediction modules. Without using task annotations beyond COCO, ECDet-S achieves 51.7 box AP with fewer than 10M parameters, while ECInsSeg-X and ECPose-X reach 48.4 mask AP and 74.8 keypoint AP, respectively. As a complementary but more compute-intensive setting, Objects365 detection pretraining consistently improves performance across all scales, with the X variants reaching 59.9 box AP, 49.8 mask AP, and 75.9 keypoint AP. These results show that compact ViTs, when combined with task-specialized distillation and edge-aware design, can be a practical and competitive solution for edge dense prediction. Code is available at: https://intellindust-ai-lab.github.io/projects/EdgeCrafter
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。