分析编译器如何用向量化、多线程和双缓冲隐藏延迟提升边缘AI性能
Analyzing Latency Hiding and Parallelism in an MLIR-based AI Kernel Compiler
- 通过MLIR管道分离向量化、多线程与双缓冲机制，量化各自贡献
- 向量化对带宽敏感核函数提升最大，多线程在调度开销摊薄后显著加速
- 双缓冲在计算与传输可重叠时有效，适用于非极致内存或计算瓶颈场景
面向边缘设备的AI内核编译依赖编译器在分层内存与显式数据移动条件下，挖掘并行性并隐藏内存延迟。本文报告了一种基准方法及在基于MLIR的编译管线中三种编译器控制机制的实测结果：向量化（Vec）、跨硬件上下文的多线程（MT）以及利用乒乓暂存缓冲区实现的双缓冲（DB），用于重叠DMA传输与计算。基于Triton/Inductor生成的内核，我们构建了消融实验，分离出Vec、MT与DB的独立贡献，并以GELU作为代表性激活函数内核，量化了MT加速比随问题规模的变化。结果显示，向量化为带宽敏感型内核带来主要收益；当调度开销被摊薄后，多线程带来显著改进；双缓冲在传输与计算可重叠时（即既非完全内存绑定也非完全计算绑定）提供额外增益。
原文摘要 · Abstract (English)
AI kernel compilation for edge devices depends on the compiler's ability to exploit parallelism and hide memory latency in the presence of hierarchical memory and explicit data movement. This paper reports a benchmark methodology and corresponding results for three compiler-controlled mechanisms in an MLIR-based compilation pipeline: vectorization (Vec), multi-threading (MT) across hardware contexts, and double buffering (DB) using ping--pong scratchpad buffers to overlap DMA transfers with compute. Using Triton/Inductor-generated kernels, we present an ablation ladder that separates the contribution of Vec, MT, and DB, and we quantify how MT speedup scales with problem size using GELU as a representative activation kernel. The results show that vectorization provides the primary gain for bandwidth-sensitive kernels, MT delivers substantial improvements once scheduling overhead is amortized, and DB provides additional benefit when transfers and compute can be overlapped (i.e., outside the extremes of purely memory-bound or purely compute-bound behavior).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。