arXiv:2606.02963cs.LG2026-06中稿 · ISCA 2026 Workshop…

用大模型自动生成跨平台高效加速器内核,提升AI推理性能。

KForge: LLM-Driven Cross-Platform Kernel Generation for AI Accelerators

论文配图:KForge: LLM-Driven Cross-Platform Kernel Generation for AI Accelerators
图 1 · 摘自论文原文
  • 双代理迭代优化:生成与性能分析交替进行,逐步逼近正确与高效。
  • 在NVIDIA B200上比TensorRT-LLM快2.12%,在Intel Arc B580上快5.13倍。
  • 适合需要快速适配多硬件的AI系统开发者和芯片厂商。

生产环境中的AI推理正面向异构加速器混合部署。智能体流水线交织推理、工具调用与多智能体协作,各阶段具有不同的计算与内存特征。为实现最优效率,每个阶段应运行在最匹配的加速器上,这带来系统挑战:需为不断增长的硬件后端与编程模型提供高性能内核。手写内核耗时且依赖底层知识,难以扩展。尽管大语言模型(LLMs)已用于自动内核生成,但低级代码生成与跨后端泛化仍存挑战。本文提出KForge,一个基于双协同代理的跨平台框架:生成代理通过编译与正确性反馈迭代生成并优化内核;性能分析代理解析程序化接口与图形化工具的剖析数据,提出优化建议以指导下一阶段合成。该循环交替执行功能修正与性能优化,最终达成接近人工调优的水平。我们在两个后端上评估:在NVIDIA B200上,相比TensorRT-LLM在gpt-oss-20b推理基准上提升2.12%端到端吞吐;在Intel Arc B580上,生成的Triton内核在KernelBench Level 2的37个GEMM+尾操作工作负载上,几何平均速度提升5.13倍,主要得益于算子融合与混合精度执行。

原文摘要 · Abstract (English)

Production inference increasingly targets a heterogeneous mix of accelerators. Agentic pipelines interleave reasoning, tool calls, and multi-agent coordination, each with distinct compute and memory profiles. For optimal efficiency, each stage should run on the accelerator best suited to it. This creates a systems challenge: each pipeline now requires high-performance kernels across a growing set of hardware backends and programming models. Writing these kernels by hand is time-consuming, demands deep low-level expertise, and does not scale as kernel complexity grows. Recently, Large Language Models (LLMs) have been leveraged for automatic kernel generation, but challenges in low-level code generation and cross-backend generalization persist. We present KForge, a cross-platform framework built around an iterative refinement loop driven by two collaborating LLM-based agents: a generation agent that produces and progressively refines kernels using compilation and correctness feedback, and a performance-analysis agent that interprets profiling data, from programmatic APIs to GUI-based tools, and emits recommendations that steer the next round of synthesis. The loop alternates between functional passes, which drive a candidate to correctness, and optimization passes, which close the performance gap to hand-tuned baselines. We evaluate KForge on two backends with very different baseline reference availability. On NVIDIA B200, KForge achieves a 2.12$\%$ improvement in end-to-end throughput compared to TensorRT-LLM on the gpt-oss-20b inference speed benchmark. On Intel Arc B580, KForge generates Triton kernels achieving a 5.13$\times$ geometric mean speedup over the faster of PyTorch eager and torch.compile on 37 GEMM + tail-ops workloads from KernelBench Level 2, primarily via operator fusion and mixed-precision execution.

内核生成大模型异构加速AI推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。