为边缘科学计算设计AI引擎部署规则,提升低延迟推理效率。
Design Rules for Extreme-Edge Scientific Computing on AI Engines

- 提出LARE指标,量化AI引擎与可编程逻辑的性能优劣边界。
- 在小批量场景下实现端到端神经网络部署,突破可编程逻辑容量限制。
- 针对低延迟需求优化数据流架构和API,适用于实时科学推理应用。
极端边缘科学应用利用机器学习模型分析传感器数据并实时决策,对延迟和吞吐量要求极高,需采用小批量处理且模型权重必须完全驻留芯片。空间数据流实现对小型网络表现良好,但因资源扩展性受限难以适应大型模型。现代FPGA SoC上的AI引擎具备高计算密度和额外片上内存,是潜在替代方案。然而其架构、编程模型及性能扩展行为与可编程逻辑存在本质差异,导致直接比较困难,使用价值不明确。本文系统地进行架构表征与微基准测试,提出延迟调整的资源等价(LARE)度量,识别出AI引擎优于可编程逻辑的场景。进一步提出面向低延迟科学推理的空间与API级数据流优化策略。最终演示了通过hlsml工具链成功部署无法放入可编程逻辑的完整神经网络。
原文摘要 · Abstract (English)
Extreme-edge scientific applications use machine learning models to analyze sensor data and make real-time decisions. Their stringent latency and throughput requirements demand small batch sizes and require that model weights remain fully on-chip. Spatial dataflow implementations are common for extreme-edge applications. Spatial dataflow works well for small networks, but it fails to scale to larger models due to inherent resource scaling limitations. AI Engines on modern FPGA SoCs offer a promising alternative with high compute density and additional on-chip memory. However, the architecture, programming model, and performance-scaling behavior of AI Engines differ fundamentally from those of the programmable logic, making direct comparison non-trivial and the benefits of using AI Engines unclear. This work addresses how and when extreme-edge scientific neural networks should be implemented on AI Engines versus programmable logic. We provide systematic architectural characterization and micro-benchmarking and introduce a latency-adjusted resource equivalence (LARE) metric that identifies when AI Engine implementations outperform programmable logic designs. We further propose spatial and API-level dataflow optimizations tailored to low-latency scientific inference. Finally, we demonstrate the successful deployment of end-to-end neural networks on AI Engines that cannot fit on programmable logic when using the hlsml toolchain.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。