将结构化剪枝与流式阵列协同设计，提升边缘设备上Transformer的推理效率。
Systolic Arrays and Structured Pruning Co-design for Efficient Transformers in Edge Systems
- 根据流式阵列尺寸定制剪枝块大小，跳过冗余计算。
- 在LibriSpeech上实现44%加速，词错误率仅上升1.4%。
- 适合资源受限的语音识别与机器翻译边缘部署场景。
在边缘设备上高效部署资源密集型Transformer模型需要跨栈优化。本文研究结构化剪枝与流式阵列加速之间的关联，将剪枝块大小与流式阵列维度对齐。在此设置下，可跳过剪枝权重块的计算，降低运行时间和能耗，但可能影响服务质量（QoS）。为评估流式阵列尺寸与稀疏性机会之间的权衡，我们提出一种新的协同设计框架，整合算法优化、系统仿真与硬件设计。以Transformer在语音识别和机器翻译中的应用为例，分析栈内配置选择对性能指标的影响。结果表明，在采用流式阵列加速的系统中，结构化剪枝能有效提升性能并维持高QoS水平。在标准LibriSpeech数据集上，结合结构化剪枝与量化，系统整体速度提升达44%，词错误率仅增加1.4%。
原文摘要 · Abstract (English)
Efficient deployment of resource-intensive transformers on edge devices necessitates cross-stack optimization. We thus study the interrelation between structured pruning and systolic acceleration, matching the size of pruned blocks with the systolic array dimensions. In this setting, computations of pruned weight blocks can be skipped, reducing run-time and energy consumption, but potentially impacting quality of service (QoS). To evaluate the trade-offs between systolic array size and sparsity opportunities, we present a novel co-design framework that integrates algorithmic optimization, system simulation, and hardware design. Targeting speech recognition and machine translation using transformers as case study, we analyze how configuration choices across the stack affect performance metrics. Results demonstrate that structured pruning on systems featuring systolic array acceleration can effectively increase performance, while maintaining high QoS levels. Up to 44% system-wide speedups due to structured pruning and quantization were measured, with only 1.4% word error rate degradation on the standard LibriSpeech dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。