让大模型自动生成高性能计算代码,通过失败反馈持续优化。
AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation

- 根据运行失败积累规则,避免重复犯错
- 在100步内实现3.12倍和1.72倍加速
- 适合需要高效生成计算核函数的开发者
近期的大语言模型代理在利用执行反馈进行测试时自适应方面展现出潜力。然而,稳健的自我改进仍远未解决:多数方法仍独立处理每个问题实例,未能积累可复用的知识。这一局限在如Triton等领域特定语言中尤为明显,这些语言在大模型预训练数据中代表性不足。其严格约束与非线性优化景观使直接生成和局部优化不可靠。我们提出AdaExplore,一个通过累积执行反馈实现性能关键核代码生成自改进的代理框架,包含两个互补阶段:失败驱动适应与多样性保持搜索,共同提升正确性与优化性能,无需额外微调或外部知识。在适应阶段,代理合成任务并将反复出现的失败转化为可复用的有效性规则记忆,帮助后续生成始终处于可行集内。在搜索阶段,代理将候选核组织为树结构,并交替进行小范围局部优化与大范围结构重生成,从而超越局部最优探索优化景观。在核运行时优化基准上的实验验证了这些提升:在100步内,AdaExplore在KernelBench Level-2和Level-3上分别实现3.12倍和1.72倍加速,并随额外计算持续改进。
原文摘要 · Abstract (English)
Recent large language model (LLM) agents have shown promise in using execution feedback for test-time adaptation. However, robust self-improvement remains far from solved: most approaches still treat each problem instance independently, without accumulating reusable knowledge. This limitation is particularly pronounced in domain-specific languages such as Triton, which are underrepresented in LLM pretraining data. Their strict constraints and non-linear optimization landscape further make naive generation and local refinement unreliable. We propose AdaExplore, an agent framework that enables self-improvement via accumulated execution feedback for performance-critical kernel code generation through two complementary stages: failure-driven adaptation and diversity-preserving search, jointly improving correctness and optimization performance without additional fine-tuning or external knowledge. In the adaptation stage, the agent synthesizes tasks and converts recurring failures into a reusable memory of validity rules, helping subsequent generations remain within the feasible set. In the search stage, the agent organizes candidate kernels as a tree and alternates between small local refinements and larger structural regeneration, allowing it to explore the optimization landscape beyond local optima. Experiments on kernel runtime optimization benchmarks validate these gains: AdaExplore achieves 3.12x and 1.72x speedups on KernelBench Level-2 and Level-3, respectively, within 100 steps, and continues to improve with additional computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。