用低秩压缩提升存内计算效率，速度最高快2.5倍、准确率提升20.9%。
Low-Rank Compression for IMC Arrays
- 采用移位复制核映射，利用空闲列并行处理，优化存内计算阵列使用。
- 提出分组低秩卷积，缓解分解矩阵中的信息失衡问题。
- 适合做模型压缩的硬件加速研究者，尤其关注存内计算场景。
本文针对存内计算（IMC）架构下的低秩模型压缩挑战展开研究。传统剪枝方法虽能减小模型尺寸，但需额外外围电路管理复杂数据流与位移问题，导致面积和能耗增加。为此，我们提出低秩压缩技术，其数据流更简洁，天然适配IMC架构。然而，该技术面临两大挑战：一是IMC阵列利用率不足，二是精度下降。为此，我们提出两种新方法：一是采用移位与复制核（SDK）映射技术，利用闲置列实现并行处理；二是引入分组低秩卷积，缓解分解矩阵中的信息不平衡。实验表明，所提方法相比现有剪枝技术最高可实现2.5倍加速或20.9%准确率提升。
原文摘要 · Abstract (English)
In this study, we address the challenge of low-rank model compression in the context of in-memory computing (IMC) architectures. Traditional pruning approaches, while effective in model size reduction, necessitate additional peripheral circuitry to manage complex dataflows and mitigate dislocation issues, leading to increased area and energy overheads. To circumvent these drawbacks, we propose leveraging low-rank compression techniques, which, unlike pruning, streamline the dataflow and seamlessly integrate with IMC architectures. However, low-rank compression presents its own set of challenges, namely i) suboptimal IMC array utilization and ii) compromised accuracy. To address these issues, we introduce a novel approach i) employing shift and duplicate kernel (SDK) mapping technique, which exploits idle IMC columns for parallel processing, and ii) group low-rank convolution, which mitigates the information imbalance in the decomposed matrices. Our experimental results demonstrate that our proposed method achieves up to 2.5x speedup or +20.9% accuracy boost over existing pruning techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。