用全局搜索优化压缩率与精度平衡，实现高效模型瘦身。
CompressNAS : A Fast and Efficient Technique for Model Compression using Decomposition
- 将秩选择视为全局优化问题，避免局部贪心
- 在ImageNet上使ResNet-18压缩8倍，准确率下降<4%
- 适合边缘设备部署，尤其对MCU和轻量NPU友好
深度卷积神经网络因规模和计算需求不断增长，难以在微控制器（MCUs）和轻量级NPU上部署。低秩张量分解（如Tucker分解）是降低参数量和计算量的有效方法，但现有方法常局部选择秩，忽略压缩与精度的全局权衡。我们提出CompressNAS，一种受MicroNAS启发的框架，将秩选择视为全局搜索问题。该框架采用快速精度评估器，可在内存与精度约束下高效探索所有候选分解方案。在ImageNet上，CompressNAS使ResNet-18压缩8倍，准确率损失低于4%；在COCO数据集上，实现YOLOv5s 2倍压缩且无精度损失，YOLOv5n压缩2倍仅损失2.5%精度。最后，我们推出新型压缩模型系列STResNet，性能优于其他高效模型。
原文摘要 · Abstract (English)
Deep Convolutional Neural Networks (CNNs) are increasingly difficult to deploy on microcontrollers (MCUs) and lightweight NPUs (Neural Processing Units) due to their growing size and compute demands. Low-rank tensor decomposition, such as Tucker factorization, is a promising way to reduce parameters and operations with reasonable accuracy loss. However, existing approaches select ranks locally and often ignore global trade-offs between compression and accuracy. We introduce CompressNAS, a MicroNAS-inspired framework that treats rank selection as a global search problem. CompressNAS employs a fast accuracy estimator to evaluate candidate decompositions, enabling efficient yet exhaustive rank exploration under memory and accuracy constraints. In ImageNet, CompressNAS compresses ResNet-18 by 8x with less than 4% accuracy drop; on COCO, we achieve 2x compression of YOLOv5s without any accuracy drop and 2x compression of YOLOv5n with a 2.5% drop. Finally, we present a new family of compressed models, STResNet, with competitive performance compared to other efficient models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。