arXiv:2510.03760cs.LGcs.AI2025-10被引 13

用大模型系统化优化CUDA内核,兼顾速度与正确性。

EvoEngineer: Mastering Automated CUDA Kernel Code Evolution with Large Language Models

  • 将内核优化形式化为有目标、约束和评估指标的任务
  • 平均提速2.72倍,代码有效率69.8%,最高达36.75倍
  • 适合需要高性能且严格保证正确性的深度学习开发者

CUDA内核优化已成为影响AI性能的关键瓶颈,深度学习训练与推理效率直接依赖高度优化的GPU内核。尽管大语言模型(LLMs)在自动化内核优化方面展现潜力,但该领域存在方法碎片化、难以比较且问题定义不清的问题。通用LLM代码演化方法也无法满足CUDA内核优化对正确性的严苛要求。为此,我们首次将CUDA内核优化形式化为具有明确目标、约束和评估指标的代码优化任务。进而构建首个系统的基于LLM的代码演化框架EvoEngineer,为优化策略的设计与适配提供指导,实现性能与正确性的平衡。最后,基于该框架实现内核优化系统,并在91个真实世界CUDA内核上进行大量实验。结果表明,EvoEngineer在性能与正确性之间达成原则性平衡,平均中位速度提升达2.72倍,代码有效性率达69.8%,优于现有方法;在所有操作中最高提速达36.75倍,50个操作中有28个(56.0%)实现超2倍加速。

原文摘要 · Abstract (English)

CUDA kernel optimization has become a critical bottleneck for AI performance, as deep learning training and inference efficiency directly depends on highly optimized GPU kernels. Despite the promise of Large Language Models (LLMs) for automating kernel optimization, this field suffers from a fragmented ecosystem of isolated and incomparable approaches with unclear problem formulations. Furthermore, general-purpose LLM code evolution methods cannot meet strict correctness requirements of CUDA kernel optimization. We address these fundamental challenges by first formalizing CUDA kernel optimization as a code optimization task with a clear objective, constraints, and evaluation metrics. We then establish the first systematic LLM-based code evolution framework, EvoEngineer, that provides guidance for designing and adapting optimization strategies to achieve a balance between performance and correctness. Finally, we implement a kernel optimization system based on this framework and conduct extensive experiments on 91 real-world CUDA kernels. Our results demonstrate that EvoEngineer achieves a principled balance between performance and correctness, with the highest averaged median speedup of \textbf{2.72}$\times$ over baseline CUDA kernels and a code validity rate of \textbf{69.8}\%, outperforming existing methods on both dimensions. Our method achieves a maximum speedup of \textbf{36.75}$\times$ among all operations over PyTorch kernels and delivers the highest speedup on \textbf{28} (\textbf{56.0\%}) of 50 operations that achieve over \textbf{2$\times$} acceleration.

CUDA优化大模型代码生成性能提升

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。