arXiv:2607.15123cs.ARcs.AI2026-07

用无ADC的模拟存内计算块,让FPGA高效运行Transformer类模型。

NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference

论文配图:NIFA: Nonlinear IMC enhanced FPGA for efficient ML inference
图 1 · 摘自论文原文
  • 用ACAM替代传统ADC,实现模拟域直接计算非线性操作。
  • 在CNN和Transformer上分别提升40倍和1.9倍能效,2.5倍面积效率。
  • 专为长序列Transformer设计,适合需要高能效推理的AI应用。

近年来,FPGA通过专用张量单元和片上内存计算提升了深度学习推理效率。基于阻变存储器(ReRAM)的模拟存内计算(IMC)进一步提升了计算密度和能效,可在ReRAM交叉阵列中直接执行向量-矩阵乘法(VMM),但传统设计仅支持静态权重的VMM,非线性运算和动态矩阵乘法(DIMM)仍需由FPGA逻辑阵列完成,导致其优势局限于静态权重模型,而依赖频繁非线性和DIMM的Transformer类模型受益有限。此外,每个IMC模块中的模数转换器(ADC)占用了超过70%的面积和功耗,限制了系统效率与可扩展性。为此,本文提出一种新型FPGA架构,采用无ADC的IMC模块,以模拟内容寻址存储器(ACAM)替代传统ADC,原生支持非线性运算。通过面向FPGA的设计空间探索,确定最优交叉阵列尺寸,在面积、灵活性与深度学习性能间取得平衡,并开发高效映射策略,利用ACAM实现DIMM,将IMC拓展至注意力计算。在CNN与Transformer基准测试中,该架构分别实现最高40倍和1.9倍能效提升,以及4.1倍和2.5倍面积效率提升。整体显著提升了FPGA深度学习推理效率,尤其在长序列输入下仍保持优异表现,推动领域专用FPGA设计发展。

原文摘要 · Abstract (English)

Recent FPGAs have improved deep learning (DL) inference efficiency through dedicated tensor blocks and in-BRAM computation. ReRAM-based analog in-memory computing (IMC) pushes efficiency further, offering an order-of-magnitude improvement in compute density and energy efficiency over conventional digital logic by performing vector-matrix multiplication (VMM) directly within the ReRAM crossbar; prior work has integrated such IMC blocks into FPGAs for DL inference. However, conventional IMC designs support only static-weight VMM, leaving nonlinear operations and dynamic matrix-matrix multiplication (DIMM) to the FPGA fabric. As a result, the benefits of IMC are largely confined to static-weight models, whereas Transformer-based models, which rely on frequent nonlinear and DIMM operations, gain only limited improvement. Moreover, the ADCs within each IMC block consume more than 70% of its area and power, further limiting system efficiency and scalability. To address these limitations, we propose a novel FPGA architecture that integrates an ADC-free IMC block, replacing the conventional ADC with analog content-addressable memories (ACAMs) that natively perform nonlinear operations inside the block. To fully exploit this block, we conduct an FPGA-aware design-space exploration that determines optimal crossbar dimensions while balancing FPGA area, flexibility, and DL performance, and we develop an efficient mapping that leverages ACAMs to carry out DIMM operations, extending the applicability of IMC to attention computation. On CNN and Transformer-based benchmarks, the proposed architecture achieves up to 40x and 1.9x higher energy efficiency and 4.1x and 2.5x higher area efficiency, respectively. Overall, it significantly improves FPGA DL inference efficiency and sustains robust gains on Transformer-based workloads across long input sequences, advancing domain-specialized FPGA design.

FPGA存内计算Transformer能效优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。