nncase统一编译框架，自动优化大模型在异构存储上的部署效率。
nncase: An End-to-End Compiler for Efficient LLM Deployment on Heterogeneous Storage Architectures
- 基于e图的重写引擎，全局搜索计算与数据移动策略。
- 在Qwen3模型上性能超越MLC LLM和Intel IPEX，接近手调llama.cpp。
- 适合需要自动化部署LLM的工程师与研究者使用。
大语言模型（LLMs）的高效部署受限于内存架构异构性，传统编译器存在流程碎片化与适配成本高的问题。我们提出nncase，一个开源的端到端编译框架，旨在统一多目标下的优化流程。核心是基于e图的项重写引擎，解决阶段顺序问题，实现计算与数据移动策略的全局探索。框架集成三个关键模块：Auto Vectorize用于适配异构计算单元，Auto Distribution通过代价感知通信优化搜索并行策略，Auto Schedule最大化片上缓存局部性。此外，缓冲区感知的Codegen阶段确保高效内核生成。评估表明，nncase在Qwen3系列模型上性能优于MLC LLM与Intel IPEX，且在CPU上达到与手调llama.cpp相当的水平，验证了自动化编译在高性能LLM部署中的可行性。源码已开源：https://github.com/kendryte/nncase。
原文摘要 · Abstract (English)
The efficient deployment of large language models (LLMs) is hindered by memory architecture heterogeneity, where traditional compilers suffer from fragmented workflows and high adaptation costs. We present nncase, an open-source, end-to-end compilation framework designed to unify optimization across diverse targets. Central to nncase is an e-graph-based term rewriting engine that mitigates the phase ordering problem, enabling global exploration of computation and data movement strategies. The framework integrates three key modules: Auto Vectorize for adapting to heterogeneous computing units, Auto Distribution for searching parallel strategies with cost-aware communication optimization, and Auto Schedule for maximizing on-chip cache locality. Furthermore, a buffer-aware Codegen phase ensures efficient kernel instantiation. Evaluations show that nncase outperforms mainstream frameworks like MLC LLM and Intel IPEX on Qwen3 series models and achieves performance comparable to the hand-optimized llama.cpp on CPUs, demonstrating the viability of automated compilation for high-performance LLM deployment. The source code is available at https://github.com/kendryte/nncase.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。