arXiv:2512.14806cs.SEcs.AI2025-12被引 2

用AI自动设计系统性能方案,效果媲美甚至超越人类专家。

Let the Barbarians In: How AI Can Accelerate Systems Performance Research

  • 通过生成-评估-优化循环,让AI自主探索系统性能解决方案。
  • 在10个案例中,AI设计的方案达到或超过人类顶尖水平。
  • 适合关注自动化系统优化的研究者和工程实践者。

人工智能正开始改变研究范式,通过自动化发现新解法。这一转变依赖可靠的验证机制,而系统性能研究天然具备此类条件:候选方案可在真实系统或模拟器中实现,并在预定义工作负载下评估。我们提出AI驱动的系统研究(ADRS)循环。基于OpenEvolve、GEPA和ShinkaEvolve等开源ADRS实例,在多区域云调度、专家混合负载均衡、基于大模型的SQL优化、事务调度等10个案例中,验证了ADRS生成的方案可匹配甚至超越人工设计的最先进方案。基于这些结果,我们总结出有效使用ADRS的最佳实践(如提示词粒度、反馈量、鲁棒评估),并讨论未来方向与影响。尽管尚未形成普适应用方法,但初步发现与识别的挑战,为研究重心转向问题定义与战略监督提供了重要指引。本文为先前工作[14]的扩展,新增多个ADRS框架的广泛评估及对最佳实践的深入分析。

原文摘要 · Abstract (English)

Artificial Intelligence (AI) is beginning to transform the research process by automating the discovery of new solutions. This shift depends on the availability of reliable verifiers, which AI-driven approaches require to validate candidate solutions. Research focused on improving systems performance is especially well-suited to this paradigm because system performance problems naturally admit such verifiers: candidates can be implemented in real systems or simulators and evaluated against predefined workloads. We term this iterative cycle of generation, evaluation, and refinement AI-Driven Research for Systems (ADRS). Using several open-source ADRS instances (i.e., OpenEvolve, GEPA, and ShinkaEvolve), we demonstrate across ten case studies (e.g., multi-region cloud scheduling, mixture-of-experts load balancing, LLM-based SQL, transaction scheduling) that ADRS-generated solutions can match or even outperform human state-of-the-art designs. Based on these findings, we outline best practices (e.g., level of prompt specification, amount of feedback, robust evaluation) for effectively using ADRS, and we discuss future research directions and their implications. Although we do not yet have a universal recipe for applying ADRS across all of systems research, we hope our preliminary findings, together with the challenges we identify, offer meaningful guidance for future work as researcher effort shifts increasingly toward problem formulation and strategic oversight. Note: This paper is an extension of our prior work [14]. It adds extensive evaluation across multiple ADRS frameworks and provides deeper analysis and insights into best practices.

AI研究系统优化自动化设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。