arXiv:2603.24517cs.LG2026-03被引 16

用智能代理取代固定算法,自动优化AI核心算子性能。

AVO: Agentic Variation Operators for Autonomous Evolutionary Search

  • 让语言模型变成自主优化的智能代理,动态调整代码。
  • 在7天内实现超cuDNN 3.5%、超FlashAttention-4 10.5%的加速效果。
  • 适合硬件优化、编译器设计及生成式AI研究者参考。

Agentic Variation Operators (AVO) 是一类新型进化变异算子,将经典进化搜索中的固定突变、交叉和人工设计启发式替换为自主编码代理。AVO 不仅生成候选代码,还通过自我导向的循环机制,结合当前代码谱系、领域知识库和执行反馈,自主提出、修复、批判与验证代码修改。我们在NVIDIA Blackwell(B200)GPU上对注意力算子进行评估,持续7天的自主演化过程中,AVO 发现的核函数在所测配置下,相比 cuDNN 最高提升3.5%,相比 FlashAttention-4 最高提升10.5%。这些优化可快速迁移至分组查询注意力,仅需30分钟额外自适应即可实现比 cuDNN 高7.0%、比 FlashAttention-4 高9.3%的性能提升。结果表明,AVO 将智能体从候选生成器升级为真正的变异算子,能发现关键微架构优化,使生成的算子在当前最先进GPU上超越专家手工设计的注意力实现。

原文摘要 · Abstract (English)

Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a language model to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware.

进化计算智能体算子优化GPU加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。