用结构与性能双重奖励,让大模型生成更优的CUDA代码。
Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation

- 引入结构感知奖励,评估内存合并、占用率等关键并行特性。
- 在C到CUDA和PyTorch到CUDA任务中,速度提升最高达5倍,正确率提高17%。
- 适合需要高性能计算代码自动优化的研究者与开发者。
强化学习结合可验证奖励(RLVR)已成为提升大模型生成优化代码能力的重要方法。然而现有方法多依赖正确性与加速比等结果信号,忽视了对程序性能至关重要的结构特征。本文提出CudaPerf,一种包含可验证执行奖励与结构化代码感知奖励的反射式强化学习框架,其奖励基于并行化特征(如内存合并、占用率、计算强度与同步模式)。该框架分两阶段运行:(1) 离线成对排序模块,通过对比学习区分强弱代码候选;(2) 在线强化学习训练,统一优化正确性、性能与结构效率。通过执行反馈进行迭代精炼,实现渐进式改进。我们构建了一个包含2.9k个C到CUDA及1k个PyTorch到CUDA程序的数据集,每个样本配有多种输入配置和多个优化策略的实现。在多个基准测试中评估,CudaPerf显著优于强基线模型(如Qwen-3-32B用于C到CUDA,CUDA Agent用于PyTorch到CUDA),分别实现最高5倍与3.32倍的加速比提升,以及17%与7%的正确率提升。
原文摘要 · Abstract (English)
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a powerful technique to enhance the reasoning capacity of LLMs for optimized code generation. However, existing RLVR approaches primarily rely on outcome-based signals such as correctness and speedup, overlooking performance-critical structural properties of programs that are essential for generating optimized code. In this work, we propose CudaPerf, a reflective RL framework that incorporates both verifiable execution rewards and structural code-aware rewards derived from parallelization features (e.g., memory coalescing, occupancy, Arithmatic Intensity, and synchronization patterns). CudaPerf operates in two stages: (1) an offline pairwise ranking module that learns to distinguish strong and weak program candidates via contrastive comparisons, and (2) an online RL training phase that jointly optimizes for correctness, performance, and structural efficiency through a unified reward signal. To further enhance learning, CudaPerf utilizes iterative refinement using execution feedback enabling progressive improvement of generated candidates. We also introduce a dataset comprising 2.9k C to CUDA and 1k PyTorch to CUDA programs, each paired with diverse input configurations and multiple CUDA implementations encompassing diverse optimization strategies. CudaPerf is evaluated across multiple benchmarks comprising both C to CUDA and PyTorch to CUDA transformations. Empirical findings suggest that CudaPerf significantly outperforms strong baselines, including Qwen-3-32B (for C to CUDA) and CUDA Agent (for PyTorch to CUDA) by achieving up to 5X & 3.32X improvements in speedup, and 17% & 7% improvements in correctness, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。