从专家代码反推优化时机，让LLM学会何时该优化
Learning When to Optimize: Verified Optimization Skills from Expert GPU-Kernel Lineages
- 通过逆向追踪专家代码的验证路径，提取可复用的优化技能
- 在两个NVIDIA架构上，优化效果和效率均超越现有基线
- 适合想提升GPU内核生成质量的编译器与AI研究者
基于大模型的智能体越来越用于生成GPU内核，但它们常知该尝试哪些优化，却不知何时优化是合理的。我们提出KLineage，从专家实现中学习这一缺失的“何时”知识：不依赖前向执行，而是通过验证门控的简化步骤逆向遍历专家代码，并将每一步有效操作转化为可复用的优化技能。每个技能不仅记录优化意图，还包含其适用代码位置、有效条件、预期效果及避免的失败情形。下游大模型在相同编译/正确性/性能门控下，将这些技能应用于新代码。在两个NVIDIA架构上的五个专家工作负载上，这些源自代码谱系的技能构成高效优化课程，在固定预算下显著优于近期基于内存的LLM内核基线。此外，我们还用22个独立测试实例进行隔离检查，以验证是否发生源例记忆。
原文摘要 · Abstract (English)
LLM-based agents are increasingly used to generate GPU kernels, but they often know what optimizations to try without knowing when those optimizations are sound. We introduce KLineage, which learns this missing "when" knowledge from expert kernels: instead of relying on forward rollouts, KLineage walks expert implementations backward through validation-gated simplifications and reverses each accepted step into a reusable optimization skill. Each skill records not only the optimization intent, but also where it applies in code, what conditions made it valid, what effect it had, and what failures its assumptions avoid. A downstream LLM materializes these skills on new code surfaces under the same compile/correctness/profile gate. On five expert workloads across two NVIDIA architectures, these lineage-derived skills serve as an effective optimization curriculum, exceeding recent memory-based LLM-kernel baselines in both final kernel quality and optimization efficiency under the same fixed budget. We additionally use a separate 22-instance held-out check as a sanity test against source-case memorization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。