arXiv:2606.11257cs.CLcs.LG2026-06被引 2

首次实现手机NPU上端到端高效运行RAG,能耗大幅降低。

Energy-Efficient On-Device RAG on a Mobile NPU: System Design and Benchmark on Snapdragon X Elite

  • 全链路神经计算部署在高通Hexagon NPU,支持嵌入、重排序与大模型生成
  • 索引阶段吞吐提升9.1倍,查询延迟降低4.0倍,能耗减少4.0倍
  • 质量与CPU/GPU相当,适合追求低功耗的移动设备边缘智能应用

检索增强生成(RAG)管道计算密集,包含嵌入、检索、重排序和大语言模型(LLM)生成。完全本地运行可提升隐私性、降低延迟并支持离线使用,但CPU推理能耗过高。本文首次实现将嵌入、重排序与LLM生成全部部署在骁龙X Elite的高通Hexagon NPU上。在戴尔XPS 13笔记本上对比了NPU、CPU及OpenCL/Adreno GPU基线。索引阶段,NPU嵌入吞吐达CPU的9.1倍,系统能耗降低12.3倍;在120个维基百科段落查询基准上,其LLM预填充速度提升18.1倍,端到端查询延迟降低4.0倍,系统能耗减少4.0倍;集成GPU比CPU慢1.7倍,能耗为NPU的6.5倍。采用GPT-4.1作为裁判评估显示,NPU答案质量与CPU/GPU在评估噪声范围内无显著差异(平均得分分别为9.32、8.95、9.03),86.7%的查询评分一致。在骁龙X Elite/Hexagon类笔记本SoC上,NPU实现了无需性能退化的节能型本地RAG,这一路径有望推广至苹果神经引擎、英特尔NPU、联发科APU等同类移动NPU。

原文摘要 · Abstract (English)

Retrieval-Augmented Generation (RAG) pipelines are compute-intensive, combining embedding, retrieval, reranking, and large language model (LLM) generation. Running them entirely on-device benefits privacy, latency, and offline use, but the energy cost of CPU inference is a major barrier. We present what is, to our knowledge, the first end-to-end RAG pipeline that runs all neural stages -- embedding, reranking, and LLM generation -- on the Qualcomm Hexagon NPU of the Snapdragon X Elite. Profiling on a Dell XPS 13 laptop, we compare NPU-accelerated RAG against CPU and OpenCL/Adreno GPU baselines on indexing and query workloads. On indexing, the NPU achieves 9.1x higher embedding throughput and 12.3x less system energy. On a 120-query Wikipedia-passage benchmark, it delivers 18.1x faster LLM prefilling, 4.0x lower end-to-end query latency, and 4.0x less system energy than the CPU baseline; the same workload on the integrated GPU is 1.7x slower than CPU and uses 6.5x more energy than the NPU. A GPT-4.1 LLM-as-judge evaluation finds NPU answer quality on par with CPU and GPU within evaluator noise (mean 9.32 vs. 8.95 vs. 9.03 on a 1-10 rubric), with 86.7% of queries scoring identically across all three backends. On the Snapdragon X Elite / Hexagon class of laptop SoC, the NPU thus enables practical, energy-efficient on-device RAG without quality regression -- a sustainable path toward green edge intelligence that we expect to generalize to comparable mobile NPUs (Apple Neural Engine, Intel NPU, MediaTek APU) as their software stacks mature.

边缘计算RAG能效优化移动NPU

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。