用可微查找表在FPGA上实现纳秒级神经网络推理,速度提升超200倍。
FPGN: Redefining Ultra-Fast Programmable Gate-based Neural Acceleration with Differentiable LUTs

- 设计可微查找表,让FPGA的查找表直接参与学习和计算。
- 实测比现有FPGA加速器快205倍,查表利用率高30倍。
- 适合追求极致低延迟的AI硬件工程师和芯片设计者。
为满足低延迟关键应用对深度神经网络(DNN)纳秒级推理的需求,本文提出FPGN,一个端到端物理感知的可微查找表(LUT)神经网络框架。传统FPGA加速器以算术运算为中心,而新兴的查表原生网络将查表视为可学习神经元,但现有方法仅限算法优化,无法实现高性能硬件部署:其可微公式不匹配FPGA查表原语,拓扑结构忽视物理布局与时序约束,且缺乏自动化设计空间探索(DSE)流程。FPGN通过三项创新解决:(i) 硬件对齐的可微训练范式,使查表神经元适配FPGA原语;(ii) 结构化查表拓扑与流式硬件架构,提升布线局部性与时序收敛;(iii) 基于高保真质量评估模型的延迟驱动编译器,自动完成DSE与硬件生成。实验表明,FPGN相比代表性基于FPGA的二值神经网络(BNN)加速器实现最高205×延迟降低,查表效率较先前可微查表网络提升30×,同时保持竞争力的推理精度。
原文摘要 · Abstract (English)
Achieving nanosecond-scale inference latency for deep neural networks (DNNs) has become a primary architectural concern for latency-critical applications. While Field-Programmable Gate Arrays (FPGAs) offer a promising substrate for low-latency inference, conventional FPGA accelerators remain arithmetic-centric, using LUTs primarily as building blocks for numerical operators and peripheral logic. In contrast, recent LUT-native neural networks treat LUTs as learnable neurons, revealing promising theoretical potential to exploit their intrinsic logic expressivity. However, existing methods are largely confined to algorithmic optimizations, failing to translate this theoretical potential into high-performance FPGA accelerators. Specifically, their differentiable formulations do not faithfully match FPGA LUT primitives, their physically-unaware topologies compromise routability and timing closure, and their lack of automated optimization flow hinders systematic design space exploration (DSE) and efficient hardware implementation. In this paper, we propose FPGN, an end-to-end physically-aware framework that closes the gap between LUT-native learning and latency-optimized FPGA implementation. FPGN addresses these challenges through (i) a hardware-aligned differentiable formulation for training FPGA-native LUT neurons, (ii) a structured LUT-native topology with a streaming hardware architecture to improve routing locality and timing closure, and (iii) a latency-driven compiler that leverages high-fidelity analytical Quality of Results models to automate DSE and hardware generation. Experiments show that FPGN achieves up to 205$\times$ latency reduction compared to representative FPGA-based BNN accelerators and up to 30$\times$ higher LUT efficiency than prior differentiable LUT-native networks, while maintaining competitive inference accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。