针对混合模型设计高效量化方法，显著降低边缘设备推理延迟。
EfficientQuant: An Efficient Post-Training Quantization for CNN-Transformer Hybrid Models on Edge Devices
- 对卷积块用统一量化，对注意力块用log2量化，兼顾精度与效率。
- 在ImageNet-1K上实现2.5到8.7倍的推理加速，精度损失极小。
- 专为边缘设备优化，适合实际部署场景中的轻量级视觉应用。
将卷积与变换器模块结合的混合模型在计算机视觉任务中表现优异，但资源开销大，难以在边缘设备部署。尽管后训练量化（PTQ）可降低资源消耗，但其在混合模型上的应用仍受限。本文提出EfficientQuant，一种结构感知的新型后训练量化方法：对卷积块采用统一量化，对变换器块采用$\log_2$量化。该方法在ImageNet-1K数据集上实现2.5×至8.7×的延迟降低，且精度损失微小。同时在边缘设备上展现出低延迟和高内存效率，具备实际部署可行性。
原文摘要 · Abstract (English)
Hybrid models that combine convolutional and transformer blocks offer strong performance in computer vision (CV) tasks but are resource-intensive for edge deployment. Although post-training quantization (PTQ) can help reduce resource demand, its application to hybrid models remains limited. We propose EfficientQuant, a novel structure-aware PTQ approach that applies uniform quantization to convolutional blocks and $log_2$ quantization to transformer blocks. EfficientQuant achieves $2.5 \times - 8.7 \times$ latency reduction with minimal accuracy loss on the ImageNet-1K dataset. It further demonstrates low latency and memory efficiency on edge devices, making it practical for real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。