用智能体+硬件反馈自动优化CUDA内核，速度快且成本低。
CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization
- 双智能体协作生成并迭代优化内核，结合NVIDIA工具实测反馈
- 平均提速1.68倍，正确率97.6%，远超现有方法
- 无需训练，跨显卡和大模型通用，单次生成仅耗时26.5分钟
高效CUDA内核对大规模LLM训练至关重要。传统手动设计耗时费力，现有自动方法常产生低效代码、开销大且泛化能力差。本文提出CudaForge，一种无需训练的多智能体工作流，模拟专家迭代优化流程：生成、验证、分析硬件反馈（如Nsight Compute指标）、再优化。采用两个LLM智能体——编码器与评审员——协同工作，通过硬件反馈持续改进。实验显示，基于OpenAI-o3等基座模型，其生成内核正确率达97.6%，相比PyTorch基线平均提速1.68倍，在KernelBench上显著超越OpenAI-o3与Kevin。CudaForge在A100、RTX 6000、4090、3090等多类GPU及多种基模型（OpenAI-o3、GPT-5、gpt-oss-120B、Claude-Sonnet-4、QwQ-32B）上均表现良好。单次优化耗时约26.5分钟，API成本约0.3美元，远低于现有方案（需6小时H100算力、5美元成本）。结果表明，无需训练的多智能体流程可实现低成本、高泛化、高性能的内核优化。
原文摘要 · Abstract (English)
Developing efficient CUDA kernels is increasingly critical for AI applications such as large-scale LLM training. However, manual kernel design is both costly and time-consuming, motivating automatic approaches that leverage LLMs for code generation. Existing methods for automatic kernel generation, however, often produce low-efficiency kernels, incur high computational overhead, and fail to generalize across settings. In this work, we propose CudaForge, a training-free multi-agent workflow for CUDA kernel generation and optimization. Our workflow is inspired by the iterative workflow of human experts, which contains steps such as developing initial kernels, testing correctness, analyzing hardware feedback, and iterative improvement. More specifically, CudaForge employs two LLM agents: a Coder and a Judge, that iteratively generate, correct, and optimize CUDA kernels, while integrating hardware feedback such as Nsight Compute (NCU) metrics. In extensive evaluations, we show that CudaForge, by leveraging base models like OpenAI-o3, achieves 97.6\% correctness of generated kernels and an average 1.68$\times$ speedup over PyTorch baselines, substantially surpassing state-of-the-art models including OpenAI-o3 and Kevin on KernelBench.Beyond accuracy and speed, CudaForge demonstrates strong generalization across GPUs (A100, RTX 6000, 4090, 3090) and base models (OpenAI-o3, GPT-5, gpt-oss-120B, Claude-Sonnet-4, QwQ-32B), while maintaining high efficiency. In particular, generating an optimized kernel takes about 26.5 minutes on one RTX6000 and incurs about \$ 0.3 API cost, which is significantly cheaper than existing agentic work that costs 6 H100 hours and \$ 5 API cost per kernel. Our results highlight that multi-agent, training-free workflows can enable cost-effective, generalizable, and high-performance CUDA kernel optimization. Code available at https://github.com/OptimAI-Lab/CudaForge
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。