arXiv:2603.10846cs.LGcs.AI2026-03被引 3

用记忆增强的强化学习,让大模型在数据少的NPU上自动写出高效代码。

Towards Cold-Start Drafting and Continual Refining: A Value-Driven Memory Approach with Application to NPU Kernel Synthesis

  • 用价值驱动的记忆检索,自动规划代码生成与优化流程。
  • 正确率从11%提升至83%,平均提速3.6倍,无需昂贵微调。
  • 适合想在冷启动环境下快速部署AI编程工具的开发者。

将大语言模型应用于数据稀缺的编程领域面临严峻挑战,尤其在新兴专用架构(如NPU)上的内核合成任务中,因缺乏训练数据形成“数据墙”。尽管模型在数据丰富的平台(如CUDA)表现优异,但在数据稀疏生态中性能急剧下降。为克服冷启动难题且避免高昂微调成本,本文提出EvoKernel——一个自演化代理框架,实现从初始代码草稿到持续优化的全流程自动化。该框架将合成过程建模为基于记忆的强化学习任务,通过新颖的价值驱动检索机制,学习各阶段的Q值,根据经验对当前目标(生成可行代码或降低延迟)的贡献度进行优先级排序。同时支持跨任务记忆共享,使代理能将简单算子的经验迁移到复杂算子。构建了NPU版KernelBench并在此评估,结果显示,EvoKernel将前沿模型的正确率从11.0%提升至83.0%,并通过迭代优化使代码中位速度提升3.60倍。这表明,价值引导的经验积累可使通用模型掌握特定硬件上的内核合成能力。

原文摘要 · Abstract (English)

Deploying Large Language Models to data-scarce programming domains poses significant challenges, particularly for kernel synthesis on emerging Domain-Specific Architectures where a "Data Wall" limits available training data. While models excel on data-rich platforms like CUDA, they suffer catastrophic performance drops on data-scarce ecosystems such as NPU programming. To overcome this cold-start barrier without expensive fine-tuning, we introduce EvoKernel, a self-evolving agentic framework that automates the lifecycle of kernel synthesis from initial drafting to continual refining. EvoKernel addresses this by formulating the synthesis process as a memory-based reinforcement learning task. Through a novel value-driven retrieval mechanism, it learns stage-specific Q-values that prioritize experiences based on their contribution to the current objective, whether bootstrapping a feasible draft or iteratively refining latency. Furthermore, by enabling cross-task memory sharing, the agent generalizes insights from simple to complex operators. By building an NPU variant of KernelBench and evaluating on it, EvoKernel improves frontier models' correctness from 11.0% to 83.0% and achieves a median speedup of 3.60x over initial drafts through iterative refinement. This demonstrates that value-guided experience accumulation allows general-purpose models to master the kernel synthesis task on niche hardware ecosystems. Our official page is available at https://evokernel.zhuo.li.

代码生成强化学习NPU

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。