COMPASS让内存计算芯片跑更大深度网络，省电又提速。
COMPASS: A Compiler Framework for Resource-Constrained Crossbar-Array Based In-Memory Deep Learning Accelerators
- 自动分层调度，让超容量网络在有限内存中运行
- 相比基线方法，吞吐提升1.78倍，能效降低28%
- 适合芯片内存受限的AI推理场景
基于交叉阵列的内存计算加速器因高吞吐和低功耗备受关注。然而现有软件与编译支持通常假设所有权重均可驻留片上，当网络规模远超内存容量时该假设失效。为解决此问题，本文提出COMPASS编译框架，专为超出片上交叉阵列容量的深度神经网络设计。该框架通过优化分层策略，将网络分割为可片上执行的多个部分，同时考虑层间数据依赖、核心利用率及写指令数量，以最小化延迟、内存访问并提升能效。仿真结果表明，相较于基线分区方法，COMPASS可在极小内存开销下支持更多网络，实现1.78倍吞吐提升，并在能量-延迟积（EDP）上获得1.28倍改善。
原文摘要 · Abstract (English)
Recently, crossbar array based in-memory accelerators have been gaining interest due to their high throughput and energy efficiency. While software and compiler support for the in-memory accelerators has also been introduced, they are currently limited to the case where all weights are assumed to be on-chip. This limitation becomes apparent with the significantly increasing network sizes compared to the in-memory footprint. Weight replacement schemes are essential to address this issue. We propose COMPASS, a compiler framework for resource-constrained crossbar-based processing-in-memory (PIM) deep neural network (DNN) accelerators. COMPASS is specially targeted for networks that exceed the capacity of PIM crossbar arrays, necessitating access to external memories. We propose an algorithm to determine the optimal partitioning that divides the layers so that each partition can be accelerated on chip. Our scheme takes into account the data dependence between layers, core utilization, and the number of write instructions to minimize latency, memory accesses, and improve energy efficiency. Simulation results demonstrate that COMPASS can accommodate much more networks using a minimal memory footprint, while improving throughput by 1.78X and providing 1.28X savings in energy-delay product (EDP) over baseline partitioning methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。