用自进化智能体系统，让大模型更高效生成高性能机器学习库代码。
Adaptive Self-improvement LLM Agentic System for ML Library Development
- 构建自适应智能体系统，通过迭代优化提升大模型生成能力。
- 在典型ML库基准上，性能最高提升3.9倍，优于单个大模型基线。
- 适合研究大模型编程、自动化机器学习库开发的开发者使用。
机器学习库通常使用面向特定架构的编程语言（ASPL）编写，这些语言针对特定硬件架构优化，是实现高效机器学习系统的关键。然而，编写高性能的ML库极具挑战性，因为需要同时掌握机器学习算法和复杂的ASPL知识。尽管大语言模型（LLMs）具备通用编码能力，但在使用它们生成基于ASPL的ML库时仍面临两大难题：一是该任务对经验丰富的程序员而言已十分复杂；二是由于ASPL本身晦涩且持续演进，可用的代码示例极少。因此，大模型需在数据有限的情况下进行复杂推理。为解决这些问题，我们提出一种自适应自我改进的智能体系统。为评估其有效性，我们在一个典型机器学习库上构建了基准测试，并使用开源与闭源大模型在此基准上生成ASPL代码。结果表明，该系统相比基线单个大模型，性能最高提升3.9倍。
原文摘要 · Abstract (English)
ML libraries, often written in architecture-specific programming languages (ASPLs) that target domain-specific architectures, are key to efficient ML systems. However, writing these high-performance ML libraries is challenging because it requires expert knowledge of ML algorithms and the ASPL. Large language models (LLMs), on the other hand, have shown general coding capabilities. However, challenges remain when using LLMs for generating ML libraries using ASPLs because 1) this task is complicated even for experienced human programmers and 2) there are limited code examples because of the esoteric and evolving nature of ASPLs. Therefore, LLMs need complex reasoning with limited data in order to complete this task. To address these challenges, we introduce an adaptive self-improvement agentic system. In order to evaluate the effectiveness of our system, we construct a benchmark of a typical ML library and generate ASPL code with both open and closed-source LLMs on this benchmark. Our results show improvements of up to $3.9\times$ over a baseline single LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。