分析NPU上因果推理算子性能，揭示长文本推理的瓶颈与优化方向。
Context-Driven Performance Modeling for Causal Inference Operators on Neural Processing Units
- 对比标准注意力与低阶变体在NPU上的表现
- 发现二次注意力内存瓶颈严重，次二次方法各有计算/访存瓶颈
- 为边缘设备高效长上下文推理提供硬件感知设计指导
大语言模型的普及推动了资源受限边缘平台上的长上下文推理需求。然而，在神经处理单元（NPUs）上部署这些模型面临显著挑战：标准注意力的二次复杂度与NPU的内存和计算模式不匹配。本文对现代NPU上的因果推理算子进行了全面性能分析，对比了二次注意力与次二次替代方案（包括结构化状态空间模型和因果卷积）。分析揭示出一系列关键瓶颈：二次注意力在长上下文场景下严重受内存限制，产生灾难性缓存效率低下；而次二次方法则在可编程向量核心上呈现计算瓶颈，或因数据移动成为内存瓶颈。这些发现为硬件感知模型与优化策略的协同设计提供了关键洞见，有助于实现边缘平台上高效的长上下文推理。
原文摘要 · Abstract (English)
The proliferation of large language models has driven demand for long-context inference on resource-constrained edge platforms. However, deploying these models on Neural Processing Units (NPUs) presents significant challenges due to architectural mismatch: the quadratic complexity of standard attention conflicts with NPU memory and compute patterns. This paper presents a comprehensive performance analysis of causal inference operators on a modern NPU, benchmarking quadratic attention against sub-quadratic alternatives including structured state-space models and causal convolutions. Our analysis reveals a spectrum of critical bottlenecks: quadratic attention becomes severely memory-bound with catastrophic cache inefficiency, while sub-quadratic variants span from compute-bound on programmable vector cores to memory-bound by data movement. These findings provide essential insights for co-designing hardware-aware models and optimization strategies to enable efficient long-context inference on edge platforms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。