让检测Transformer提速50%且保持高分辨率精度
Cross Resolution Encoding-Decoding For Detection Transformers
- 提出跨分辨率编码解码机制，融合多尺度特征
- 在MS-COCO上实现202G FLOPs的高精度，仅需一半计算量
- 适合追求高效高精度目标检测的开发者使用
检测变压器（DETR）是知名的目标检测框架，但实现计算高效的多尺度检测仍具挑战。本文提出一种跨分辨率编码解码（CRED）机制，使DETR在保持高分辨率检测精度的同时，达到低分辨率检测的速度。CRED包含两个模块：跨分辨率注意力模块（CRAM）用于将低分辨率编码器输出的知识传递至高分辨率特征；一步多尺度注意力（OSMA）则在单步内融合多尺度特征，生成富含多尺度信息的目标特征图。在主流DETR方法中应用CRED后，其性能接近高分辨率模型，但计算量减少约50%。例如，最先进的DN-DETR在采用CRED（称为CRED-DETR）后，速度提升76%，总计算量降至约202G FLOPs（原为400+G），显著优于高分辨率版本。作者计划向社区公开预训练的CRED-DETR模型。
原文摘要 · Abstract (English)
Detection Transformers (DETR) are renowned object detection pipelines, however computationally efficient multiscale detection using DETR is still challenging. In this paper, we propose a Cross-Resolution Encoding-Decoding (CRED) mechanism that allows DETR to achieve the accuracy of high-resolution detection while having the speed of low-resolution detection. CRED is based on two modules; Cross Resolution Attention Module (CRAM) and One Step Multiscale Attention (OSMA). CRAM is designed to transfer the knowledge of low-resolution encoder output to a high-resolution feature. While OSMA is designed to fuse multiscale features in a single step and produce a feature map of a desired resolution enriched with multiscale information. When used in prominent DETR methods, CRED delivers accuracy similar to the high-resolution DETR counterpart in roughly 50% fewer FLOPs. Specifically, state-of-the-art DN-DETR, when used with CRED (calling CRED-DETR), becomes 76% faster, with ~50% reduced FLOPs than its high-resolution counterpart with 202 G FLOPs on MS-COCO benchmark. We plan to release pretrained CRED-DETRs for use by the community. Code: https://github.com/ashishkumar822/CRED-DETR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。