让FPGA轻松跑LSTM,支持混合精度,省电又高效。
FINN-GL: Generalized Mixed-Precision Extensions for FPGA-Accelerated LSTMs
- 用ONNX的Scan算子模拟LSTM循环结构,支持混合量化。
- 在XCZU7EV上生成量化ConvLSTM硬件,延迟低且资源少。
- 适合做低功耗实时时序预测的工程师和研究者。
循环神经网络(RNN)特别是长短期记忆网络(LSTMs)在情感分析、短期股票预测等时序任务中表现优异。但其计算复杂度限制了在资源受限环境中的实时部署。尽管FPGA是能效高的AI加速平台,现有工具多聚焦前馈网络,LSTM加速通常需全定制实现。本文基于开源可扩展的FINN框架,提出通用LSTM FPGA部署方案。通过ONNX规范中的Scan算子建模LSTM的递归特性,支持模型内部混合量化并实现功能验证。进一步在FINN编译器中引入自定义变换,将量化后的ONNX计算图映射至FINN HLS核库与Vitis HLS的硬件模块。我们使用广泛采用的数据集训练了一个量化ConvLSTM模型,并通过该流程为目标设备XCZU7EV生成对应硬件IP。结果表明,生成的量化ConvLSTM加速器在延迟与资源消耗间取得良好平衡,同时推理精度匹配甚至优于当前主流高精度模型。我们认为该方法具有普适性,将推动FPGA上高效RNN加速器的设计。
原文摘要 · Abstract (English)
Recurrent neural networks (RNNs), particularly LSTMs, are effective for time-series tasks like sentiment analysis and short-term stock prediction. However, their computational complexity poses challenges for real-time deployment in resource constrained environments. While FPGAs offer a promising platform for energy-efficient AI acceleration, existing tools mainly target feed-forward networks, and LSTM acceleration typically requires full custom implementation. In this paper, we address this gap by leveraging the open-source and extensible FINN framework to enable the generalized deployment of LSTMs on FPGAs. Specifically, we leverage the Scan operator from the Open Neural Network Exchange (ONNX) specification to model the recurrent nature of LSTM computations, enabling support for mixed quantisation within them and functional verification of LSTM-based models. Furthermore, we introduce custom transformations within the FINN compiler to map the quantised ONNX computation graph to hardware blocks from the HLS kernel library of the FINN compiler and Vitis HLS. We validate the proposed tool-flow by training a quantised ConvLSTM model for a mid-price stock prediction task using the widely used dataset and generating a corresponding hardware IP of the model using our flow, targeting the XCZU7EV device. We show that the generated quantised ConvLSTM accelerator through our flow achieves a balance between performance (latency) and resource consumption, while matching (or bettering) inference accuracy of state-of-the-art models with reduced precision. We believe that the generalisable nature of the proposed flow will pave the way for resource-efficient RNN accelerator designs on FPGAs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。