用代码图结构提升算法自动设计效率,节省计算资源。
Budget-Efficient Automatic Algorithm Design via Code Graph

- 将算法拆解为可组合的代码修正操作,按需生成新算法。
- 在相同计算预算下,比传统全算法搜索性能更优,提升23%以上。
- 适合需要高效生成算法的开发者或研究者,尤其关注资源约束场景。
大型语言模型(LLM)已成为自动算法设计的强大工具,但现有流程效率低下:以完整算法为单位进行搜索,重复编写通用子结构,并丢弃可能包含有价值特征的低适配度候选。本文提出预算高效的自动算法设计范式,即在有限计算成本下最大化实际适配度。我们采用有向无环图表示算法,构建搜索框架,充分利用LLM输出。不再直接请求完整算法,而是让LLM提供紧凑的代码修正操作——添加、替换或删除代码块。每次修正扩展图结构,生成可组合的新算法。该图结构将算法分解为修正集合,实现修正级别的信用分配,指导后续查询。我们还提供了关于不同预算水平下搜索深度与广度理想平衡的理论分析。在三个组合优化问题上验证,本方法在同等令牌预算下显著优于全算法搜索。实验还表明,丰富上下文仅在模型先验知识不足时有效,否则反而降低性能。
原文摘要 · Abstract (English)
Large language models (LLMs) have emerged as powerful tools for automatic algorithm design (AAD). However, existing pipelines remain inefficient. They operate at the granularity of full algorithms, redundantly rewriting recurring substructures and discarding low-fitness candidates that may contain valuable algorithmic features. We formalize budget-efficient automatic algorithm design, wherein the search policy maximizes realized fitness subject to limited computational cost. We propose a directed acyclic graph representation of algorithms and build a search framework that fully exploits the LLM's output. Instead of querying the LLM for full algorithms, we use it to obtain corrections: compact operators that add, replace, or remove code blocks. Each correction augments the graph, yielding new algorithms that compose with prior corrections. This graph structure decomposes algorithms into sets of corrections, enabling correction-level credit assignment that informs subsequent queries. We complement this framework with theoretical insights into the ideal balance between search depth and breadth at different budget levels. We validate our method empirically on three combinatorial optimization problems, demonstrating consistent superiority of our graph-based search over full-algorithm search at equal token budget. Finally, our experiments suggest that rich contexts help only when the LLM's prior knowledge is shallow, and can hinder performance otherwise.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。