DICE用扩散模型生成高性能CUDA核，效果超越现有方法。
DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels
- 采用分阶段强化学习训练扩散模型，先补全后生成完整CUDA核。
- 在KernelBench上表现超越同规模自回归与扩散模型，最优达80.3%正确率。
- 专为CUDA核设计数据集和训练框架，适合性能敏感的代码生成场景。
扩散型大语言模型（dLLMs）因其并行生成能力，成为代码生成的有力候选，尤其适合需要整体结构规划与非顺序优化的场景。然而，针对CUDA核生成的定制仍面临高专业性与高质量训练数据匮乏的挑战。为此，我们构建了CuKe数据集，专门用于高性能CUDA核的监督微调。在此基础上，提出双阶段精心设计的强化学习框架（BiC-RL），包含核补全与端到端生成两个阶段。基于此框架，我们推出DICE系列扩散大模型，涵盖1.7B、4B和8B三种参数量级。在KernelBench上的大量实验表明，DICE显著优于同等规模的自回归与扩散模型，确立了当前CUDA核生成的新基准。
原文摘要 · Abstract (English)
Diffusion large language models (dLLMs) have emerged as a compelling alternative to autoregressive (AR) LLMs, owing to their capacity for parallel token generation. This paradigm is particularly well-suited for code generation, where holistic structural planning and non-sequential refinement are critical. Despite this potential, tailoring dLLMs for CUDA kernel generation remains challenging, obstructed not only by the high specialization but also by the severe lack of high-quality training data. To address these challenges, we construct CuKe, an augmented supervised fine-tuning dataset optimized for high-performance CUDA kernels. On top of it, we propose a bi-phase curated reinforcement learning (BiC-RL) framework consisting of a CUDA kernel infilling stage and an end-to-end CUDA kernel generation stage. Leveraging this training framework, we introduce DICE, a series of diffusion large language models designed for CUDA kernel generation, spanning three parameter scales, 1.7B, 4B, and 8B. Extensive experiments on KernelBench demonstrate that DICE significantly outperforms both autoregressive and diffusion LLMs of comparable scale, establishing a new state-of-the-art for CUDA kernel generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。