arXiv:2606.07586cs.LGcs.AI2026-06中稿 · the Machine Learni…被引 1

用智能体系统自动部署大模型到边缘NPU,省时高效且精度可靠。

From Human Guidance to Autonomy: Agent Skill System for End-to-End LLM Deployment on Spatial NPUs

论文配图:From Human Guidance to Autonomy: Agent Skill System for End-to-End LLM Deployment on Spatial NPUs
图 1 · 摘自论文原文
  • 分两阶段:先人工辅助开发参考模型,再提炼为可复用的智能体技能体系。
  • 8个新模型在AMD XDNA 2 NPU上0.5~4小时完成部署,速度提升2.2倍以上。
  • 全程无需人工干预,适合资源受限场景下的大模型边缘落地。

空间神经处理器(Spatial NPUs)为边缘端大模型推理提供了能效优势,但实现端到端部署仍高度依赖人工。现有研究多聚焦单核优化,缺乏对资源受限空间NPUs上完整模型部署的支持。本文提出一种两阶段方法,基于AMD XDNA 2 NPU,从人工引导逐步过渡至智能体自治。第一阶段通过人机协作实现Llama-3.2-1B的参考部署,在prefill阶段提速2.2倍,decode阶段提速4.0倍,同时记录优化轨迹并生成结构化文档。第二阶段将文档提炼为包含八个阶段的智能体技能系统,严格保证每阶段数值正确性。利用该系统,我们成功在开放编译栈下自主部署8个解码器型LLM(Llama-3.2-3B、SmolLM2-1.7B、Qwen2.5-{0.5B, 1.5B, 3B}、Qwen3-{0.6B, 1.7B, 4B}),此前未有公开软件栈实现过这些模型在AMD NPUs上的部署。每个部署耗时0.5–4小时,几乎无需人工介入,均通过数值正确性验证,证明了对未见模型的功能泛化能力。其中三个模型性能达到或超过参考部署水平,表明无需额外人工调优即可获得竞争力表现。

原文摘要 · Abstract (English)

Spatial neural processing units (NPUs) provide an energy-efficient platform for edge LLM inference, but efficiently deploying an LLM end-to-end on such hardware remains labor-intensive. Although AI coding agents have begun to lower this cost, existing studies have largely focused on single-kernel optimization rather than end-to-end LLM deployment on resource-constrained spatial NPUs. We present a two-stage methodology, instantiated on the AMD XDNA 2 NPU, that progresses from human-guided development to agent autonomy. In the first stage, we develop a reference deployment of Llama-3.2-1B through human-guided agent assistance. The resulting implementation achieves a speedup of 2.2x on prefill and 4.0x on decode over the hand-optimized baseline, with the optimization trajectory and its lessons recorded as structured documentation throughout. In the second stage, we distill the documentation into an agent skill system consisting of eight phases, orchestrating the optimization and debugging skill sets, with numerical correctness strictly enforced at each phase. Using our agent skill system, we autonomously deploy eight additional decoder-only LLMs (Llama-3.2-3B, SmolLM2-1.7B, Qwen2.5-{0.5B, 1.5B, 3B}, Qwen3-{0.6B, 1.7B, 4B}) end-to-end on the AMD XDNA 2 NPU using the open-source compiler stack. To our knowledge, these models have not previously been deployed on AMD NPUs via any open-source software stack. Each deployment completes in 0.5-4 hours of agent wall time with almost no human guidance, and passes the numerical-correctness gates, demonstrating functional generalization to previously unencountered LLMs. Three of the eight match or exceed the sustained performance of our Llama-3.2-1B reference deployment, suggesting that the resulting implementations can be competitive without additional model-specific human engineering.

边缘计算智能体大模型部署NPU优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。