arXiv:2607.09385cs.DCcs.AI2026-07中稿 · IEEE COINS 2026

针对AMD NPU优化长序列推理,实现节能9倍以上。

STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU

论文配图:STEEL: Sparsity-Aware Fused Attention for Energy-Efficient Long-Sequence Inference on AMD's XDNA NPU
图 1 · 摘自论文原文
  • 设计新数据流架构,利用NPU片上内存与并行性提升效率。
  • 通过稀疏感知流水线减少同步开销,延迟降低9.6倍,速度提升22.8倍。
  • 开源首个适配XDNA架构的FlashAttention实现,适合边缘端大模型部署。

大型语言模型代理在操作系统工作流中的广泛应用,凸显了笔记本级系统芯片(SoC)上节能推理的重要性。尽管云迁移仍普遍,但其带来的可靠性与隐私问题对代理类任务尤为突出。近年笔记本SoC集成神经处理引擎(NPUs)以优化能效,但因架构多样性和显式数据移动编程模型,注意力机制的有效映射仍具挑战。本文提出STEEL,首个面向XDNA类NPU的FlashAttention开源实现。STEEL引入预填充注意力的数据流公式,高效利用空间并行与片上内存。同时,通过稀疏感知流水线布局缓解因果掩码导致的负载不均,降低同步开销并提升利用率。我们在AMD Ryzen AI 9 HX 370 SoC上评估STEEL,对比优化后的CPU与GPU实现。实验显示,相比CPU基线,能量消耗平均降低9.17倍;相比GPU基线,降低1.75倍。在XDNA 1上,较前序最优方案平均延迟减少9.6倍;在XDNA 2上,相较逐层注意力实现平均提速22.8倍。

原文摘要 · Abstract (English)

The growing adoption of large language model-based agents within operating system workflows has increased the importance of energy-efficient inference on laptop-class systems-on-chip (SoCs). While cloud offloading remains common, it introduces reliability and privacy concerns that are particularly problematic for agentic workloads. Recent laptop SoCs, therefore, incorporate neural processing engines (NPUs) optimized for energy efficiency; however, effectively mapping attention mechanisms onto NPUs remains challenging due to architectural diversity and explicit data-movement programming models. In this work, we present STEEL, the first open-source implementation of FlashAttention targeting XDNA-like NPUs. STEEL introduces a dataflow formulation of prefill attention, enabling efficient exploitation of spatial parallelism and on-chip memory. Furthermore, STEEL addresses the load imbalance induced by the causal mask by leveraging a sparsity-aware pipeline placement onto the NPU array, reducing synchronization overhead and improving utilization. We evaluate STEEL on the AMD Ryzen AI 9 HX 370 SoC and compare its performance against optimized CPU and GPU implementations. Experimental results show that STEEL reduces energy consumption by an average of 9.17x and 1.75x relative to CPU and GPU baselines, respectively. On XDNA 1, STEEL achieves an average 9.6x latency reduction over the prior state of the art, and delivers a 22.8x speedup on average compared to a layer-by-layer attention implementation on XDNA 2.

NPU优化长序列推理节能计算FlashAttention

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。