支持动态混合精度的内存计算芯片,推理更高效节能。
BF-IMNA: A Bit Fluid In-Memory Neural Architecture for Neural Network Acceleration
- 用可变位宽的存内计算单元实现软硬件协同的混合精度推理。
- 在不同延迟约束下,精度与能效权衡明显,最低延迟时能效最优。
- 无需重配置即可适配多种模型,适合部署于资源受限的边缘设备。
混合精度量化使神经网络在硬件上实现更高效,提升吞吐量并降低能耗。存内计算(IMC)架构通过数据驱动的计算范式缓解内存墙问题,具备高吞吐与高能效。然而现有IMC加速器仅支持静态定点精度,难以灵活适配混合精度网络。本文提出BF-IMNA,一种面向端到端卷积神经网络推理的比特流存内神经架构,可在运行时无硬件重配置开销下支持静态与动态混合精度。核心为关联处理器(APs),即位串行字并行的单指令多数据(SIMD)类引擎。我们评估了在不同技术(eNVM与NVM)、混合精度配置及供电电压条件下,基于ImageNet数据集对AlexNet、VGG16和ResNet50的端到端推理性能。为验证比特流能力,我们在BF-IMNA上实现HAWQ-V3的逐层混合精度配置于ResNet18,设置不同延迟预算。结果表明:高延迟约束下,混合精度接近固定精度INT8精度,但能量延迟积(EDP)更差;低延迟约束下,其EDP最接近固定精度INT4,精度下降更显著。此外,固定精度配置下,BF-IMNA仍达当前顶尖加速器水平:能效提升20%,吞吐率提高2%。
原文摘要 · Abstract (English)
Mixed-precision quantization works Neural Networks (NNs) are gaining traction for their efficient realization on the hardware leading to higher throughput and lower energy. In-Memory Computing (IMC) accelerator architectures are offered as alternatives to traditional architectures relying on a data-centric computational paradigm, diminishing the memory wall problem, and scoring high throughput and energy efficiency. These accelerators can support static fixed-precision but are not flexible to support mixed-precision NNs. In this paper, we present BF-IMNA, a bit fluid IMC accelerator for end-to-end Convolutional NN (CNN) inference that is capable of static and dynamic mixed-precision without any hardware reconfiguration overhead at run-time. At the heart of BF-IMNA are Associative Processors (APs), which are bit-serial word-parallel Single Instruction, Multiple Data (SIMD)-like engines. We report the performance of end-to-end inference of ImageNet on AlexNet, VGG16, and ResNet50 on BF-IMNA for different technologies (eNVM and NVM), mixed-precision configurations, and supply voltages. To demonstrate bit fluidity, we implement HAWQ-V3's per-layer mixed-precision configurations for ResNet18 on BF-IMNA using different latency budgets, and results reveal a trade-off between accuracy and Energy-Delay Product (EDP): On one hand, mixed-precision with a high latency constraint achieves the closest accuracy to fixed-precision INT8 and reports a high (worse) EDP compared to fixed-precision INT4. On the other hand, with a low latency constraint, BF-IMNA reports the closest EDP to fixed-precision INT4, with a higher degradation in accuracy compared to fixed-precision INT8. We also show that BF-IMNA with fixed-precision configuration still delivers performance that is comparable to current state-of-the-art accelerators: BF-IMNA achieves $20\%$ higher energy efficiency and $2\%$ higher throughput.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。