通过提前规划数据与指令流，实现高效低延时的AI推理执行
Hardware-Aware Data and Instruction Mapping for AI Tasks: Balancing Parallelism, I/O and Memory Tradeoffs
- 提前生成统一指令与数据流，减少主机干预和外部内存访问
- 在VGG-19上实现88%-92%硬件利用率，97%消息本地生成，近89%时间在片上完成传输
- 适合追求高吞吐、低功耗的边缘AI芯片设计者
我们提出一种面向深度学习推理的映射框架，利用神经网络行为的可预测性，提前规划计算与通信。该框架生成统一的数据与指令流，使硬件可自主执行操作并路由信息，大幅降低对主机调度和片外内存的依赖。基于可编程的消息传递架构，通过权重驻留复用、片上广播和分阶段聚合等技术，保持数据移动局部化，并协调阵列内计算。应用于VGG-19时，硬件利用率维持在88%至92%之间，超过97%的消息由内部生成，近89%的时间消耗在片上传输。更大规模阵列下计算吞吐突破1 TFLOP/s，每层通过复用与聚合实现高达100 MB的通信量削减。结果表明，基于流式计算的执行方式有效，而本映射器通过紧密协同数据与指令流，成功支撑了这一执行模式。
原文摘要 · Abstract (English)
We introduce a mapping framework for deep learning inference that takes advantage of predictable neural network behavior to plan both computation and communication ahead of time. The framework generates a unified stream of instructions and data, enabling the hardware to execute operations and route information on its own, without frequent involvement from the host and with minimal off-chip memory use. This naturally reduces reliance on I/O, off-chip memory, and host control. By leveraging fine-grained message passing on a programmable, message-based compute architecture, the framework keeps data movement local and coordinates computation across the array using techniques such as stationary-weight reuse, in-array multicasting, and staged reductions. Applied to VGG-19, the framework sustains high utilization (88 to 92 percent), with over 97 percent of messages generated internally and nearly 89 percent of time consumed on-chip transfers. Computation throughput scales beyond 1 TFLOP/s on larger arrays, while traffic reductions from reuse and local aggregation reach up to 100 MB per layer. Overall, the results highlight the effectiveness of streaming-based computation and show how our mapper enables this execution style by tightly coordinating data and instruction flow across the hardware.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。