联合优化模型架构、量化精度与硬件加速,提升边缘设备部署效率。
JAQ: Joint Efficient Architecture Design and Low-Bit Quantization with Hardware-Software Co-Exploration
- 通过通道级稀疏量化降低训练内存开销。
- 在ImageNet上比先前方法高7%的准确率,每轮硬件搜索仅需0.15秒。
- 适合资源受限场景下的高效模型与硬件协同设计。
神经网络架构、量化精度与硬件加速器的协同设计,为在资源受限的边缘设备上实现性能与效率的最优平衡提供了可行路径。本文提出JAQ框架,联合优化这三个关键维度。然而,在大规模搜索空间中自动化该过程仍面临显著挑战,尤其在极低比特量化时。主要挑战包括:(1)软件侧内存开销:低精度量化感知训练会导致大量中间特征和潜在权重的存储,引发内存耗尽;(2)硬件侧搜索耗时:硬件参数离散且编译优化与各算子间相互作用复杂,导致加速器搜索耗时。为此,JAQ采用通道级稀疏量化(CSQ)方案,仅对模型中最敏感部分应用量化以缓解内存压力。同时,设计了BatchTile,利用硬件生成网络编码所有可能的分块模式,显著加快编译映射策略的搜索速度。大量实验表明,JAQ在ImageNet上相比先前方法实现约7%的Top-1准确率提升,且每轮硬件搜索时间缩短至0.15秒。
原文摘要 · Abstract (English)
The co-design of neural network architectures, quantization precisions, and hardware accelerators offers a promising approach to achieving an optimal balance between performance and efficiency, particularly for model deployment on resource-constrained edge devices. In this work, we propose the JAQ Framework, which jointly optimizes the three critical dimensions. However, effectively automating the design process across the vast search space of those three dimensions poses significant challenges, especially when pursuing extremely low-bit quantization. Specifical, the primary challenges include: (1) Memory overhead in software-side: Low-precision quantization-aware training can lead to significant memory usage due to storing large intermediate features and latent weights for back-propagation, potentially causing memory exhaustion. (2) Search time-consuming in hardware-side: The discrete nature of hardware parameters and the complex interplay between compiler optimizations and individual operators make the accelerator search time-consuming. To address these issues, JAQ mitigates the memory overhead through a channel-wise sparse quantization (CSQ) scheme, selectively applying quantization to the most sensitive components of the model during optimization. Additionally, JAQ designs BatchTile, which employs a hardware generation network to encode all possible tiling modes, thereby speeding up the search for the optimal compiler mapping strategy. Extensive experiments demonstrate the effectiveness of JAQ, achieving approximately 7% higher Top-1 accuracy on ImageNet compared to previous methods and reducing the hardware search time per iteration to 0.15 seconds.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。