用智能体进化框架自动生成昇腾硬件算子,提升推理性能。
AgenticCANN: Automated Ascend C Operator Generation via Knowledge-Augmented Agentic Evolution

- 构建知识增强的智能体演化系统,解决昇腾算子开发知识不足问题。
- 在六类算子上实现56%至100%可行性,最大提速6.65倍。
- 适合需要高效部署NPU算子的模型开发者和硬件优化工程师。
昇腾C算子优化对NPU推理性能至关重要,但需深厚硬件知识。尽管大语言模型在自动化CUDA内核生成方面表现良好,但昇腾C的编程模型差异带来了独特挑战,尚未被充分探索。本文提出AgenticCANN,一种专为低语料环境下的昇腾算子合成设计的知识增强型智能体演化框架。为克服陌生硬件平台的知识匮乏,该框架引入知识协调生成系统,在开发全周期提供多层级领域洞察,解决上游可行性瓶颈。在此基础上,采用阶段自适应的智能体演化策略,动态匹配不同生成与演进阶段的LLM交互模式,平衡高探索性候选发现与高性能收敛调优。在华为昇腾910B上对六类跨五种模式的算子进行大量实验,结果表明:逐元素和归一化算子可行性达90%~100%,融合算子达56%,1B Pangu模型推理内核最高提速6.65倍。进一步分析显示,知识注入使逐元素算子可行性从57%提升至86%,证明其具有普遍性而非算子特异性优势。
原文摘要 · Abstract (English)
Ascend C operator optimization is critical for NPU (Neural Processing Unit) inference performance but requires deep hardware expertise. While large language models (LLMs) have shown promise in automated CUDA kernel generation, the fundamentally different programming model of Ascend C introduces unique challenges that remain unexplored. In this paper, we propose AgenticCANN, a knowledge-augmented agentic evolution framework specifically tailored for automated Ascend C operator synthesis in low-corpus NPU environments. To overcome the severe platform knowledge deficit on unfamiliar hardware, AgenticCANN incorporates a knowledge-orchestrated generation system that delivers structured, multi-level domain insights across the development lifecycle to resolve the upstream feasibility bottleneck. Building on this foundation, it features a stage-adaptive agentic evolution strategy that dynamically aligns LLM interaction modes with specific generation and evolution phases, balancing high-exploration candidate discovery with high-convergence performance tuning. Extensive experiments on Huawei Ascend 910B across six operators spanning five pattern categories demonstrate that our method achieves 90 to 100 percent feasibility on elementwise and normalization operators, 56% on fusion operators, and up to 6.65$\times$ speedup on 1B Pangu model inference kernels. Further analysis reveals that knowledge injection monotonically improves feasibility from 57% to 86% on elementwise operators, demonstrating its general rather than operator-specific benefit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。