arXiv:2605.31464cs.LGcs.AI2026-05

用大模型当智能裁判,减少GPU实测次数,更快找到高效代码。

GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization

论文配图:GPU Forecasters: Language Models as Selective Surrogates for Kernel Runtime Optimization
图 1 · 摘自论文原文
  • 让大模型预测代码性能,只在它有把握时才用真实GPU测试。
  • 在相同测试次数下,能探索的代码候选数多3倍以上。
  • 适合做深度学习优化、想加速硬件调优的研究者和工程师。

GPU内核是现代深度学习的核心,优化它们通常需在目标硬件上反复测量。然而每次评估都需编译并执行,成本高昂。随着大语言模型(LLM)推理成本下降,基于LLM的搜索可扩展至更大范围,但设备端评估成为瓶颈。本文研究如何让LLM作为选择性代理,预测内核性能。理想代理应准确且具备自我判断能力,知道何时可能出错并转交真实测试。我们评估其预测准确性、校准度及在有限GPU预算下发现高性能内核的实际价值。实验表明,LLM可准确预测相对性能,通过强化学习可进一步提升准确性和置信度校准。在内核搜索中使用该代理后,在相同GPU评估预算下可探索数倍于基线的候选方案,从而发现更优内核。结果表明,LLM可超越单纯生成代码的角色,作为虚拟GPU模型参与优化全过程。

原文摘要 · Abstract (English)

GPU kernels are the workhorse of modern deep learning, and optimizing them (via evolutionary search or coding agents) usually requires repeated measurement on target hardware. While these measurements provide the ground-truth signal necessary for kernel search, they are costly, because each evaluation of a kernel requires compilation and repeated execution on a GPU. As improvements in LLM inference reduce the cost of writing novel kernels and LLM-driven searches scale to large search budgets, on-device evaluation becomes a bottleneck. To address this, we study how LLMs can serve as selective GPU surrogates for kernel evaluation, by forecasting the performance of proposed kernels. A useful surrogate should be accurate, and it should be selective, by knowing when it could be wrong, and deferring to the GPU. To evaluate surrogates, we measure whether their forecasts are accurate, calibrated, and practically useful for recovering fast kernels under limited GPU-measurement budgets. Next, we study whether reinforcement learning can improve forecast accuracy and confidence calibration. Our experiments demonstrate that LLMs can accurately forecast relative kernel performance, that their utility can be improved through reinforcement learning. Used inside a kernel search, the surrogate lets the search consider several times as many candidates under the same GPU evaluation budget, and that leads to finding faster kernels than an equal-budget baseline. These results suggest that LLMs can play a broader role in kernel optimization, by acting as virtual models of a GPU rather than solely as kernel generators for search.

GPU优化大模型应用代码生成强化学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。