通过像素熵聚类减少检测变压器计算量，精度损失小。
ENACT: Entropy-based Clustering of Attention Input for Reducing the Computational Needs of Object Detection Transformers
- 基于输入像素熵相似性进行聚类，降低注意力计算复杂度。
- 在COCO数据集上内存占用下降，精度仅轻微降低。
- 可作为插件模块适配多种检测变压器，适合资源受限场景。
Transformer在视觉目标检测任务中表现出优异的精度，但其注意力权重的平方级规模导致计算开销巨大。本文提出基于输入熵的聚类方法，利用相同物体像素间熵的相似性，降低计算需求。该思想通过名为ENACT（Entropy-based Clustering of Attention Input for Detection Transformers）的模块实现，可作为任意多头自注意力架构的即插即用组件。在COCO目标检测数据集及三种检测Transformer上的实验表明，该方法显著降低了内存占用，同时精度下降可控。代码已开源：https://github.com/GSavathrakis/ENACT。
原文摘要 · Abstract (English)
Transformers demonstrate competitive performance in terms of precision on the problem of vision-based object detection. However, they require considerable computational resources due to the quadratic size of the attention weights. In this work, we propose to cluster the transformer input on the basis of its entropy, due to its similarity between same object pixels. This is expected to reduce GPU usage during training, while maintaining reasonable accuracy. This idea is realized with an implemented module that is called ENtropy-based Attention Clustering for detection Transformers (ENACT), which serves as a plug-in to any multi-head self-attention based transformer network. Experiments on the COCO object detection dataset and three detection transformers demonstrate that the requirements on memory are reduced, while the detection accuracy is degraded only slightly. The code of the ENACT module is available at https://github.com/GSavathrakis/ENACT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。