用代码结构反馈指导大模型生成优化算法,提速且效果更好
LLaMEA-SAGE: Guiding Automated Algorithm Design with Structural Feedback from Explainable AI
- 从代码抽象语法树提取结构特征,生成自然语言指令引导算法演化
- 在小实验中达到相同性能速度更快,在大模型基准上优于现有方法
- 适合研究自动化算法设计、可解释AI与大模型协同的学者
大语言模型已能通过自然语言提示直接生成优化算法。尽管基于进化框架的LLaMEA在算法设计空间中具有强大的探索能力,但其搜索过程仅依赖适应度反馈,未利用生成代码中的丰富信息。本文提出一种机制,基于从生成算法的抽象语法树中提取的图论和复杂性特征,构建代理模型并学习历史解档案,结合可解释AI识别影响性能的关键特征,将其转化为自然语言突变指令,以指导后续基于大模型的代码生成,同时保持表达自由度。我们提出LLaMEA-SAGE,将这种特征驱动的引导集成到LLaMEA中,并在多个基准上进行评估。小规模实验表明,所提结构化引导在达成相同性能时速度更快;在更大规模的GECCO-MA-BBOB竞赛套件(MA-BBOB)实验中,该方法优于当前最先进的自动算法设计方法。结果表明,代码衍生信号可有效引导大模型驱动的算法演化,弥合了代码结构与人类可理解性能反馈之间的鸿沟。
原文摘要 · Abstract (English)
Large language models have enabled automated algorithm design (AAD) by generating optimization algorithms directly from natural-language prompts. While evolutionary frameworks such as LLaMEA demonstrate strong exploratory capabilities across the algorithm design space, their search dynamics are entirely driven by fitness feedback, leaving substantial information about the generated code unused. We propose a mechanism for guiding AAD using feedback constructed from graph-theoretic and complexity features extracted from the abstract syntax trees of the generated algorithms, based on a surrogate model learned over an archive of evaluated solutions. Using explainable AI techniques, we identify features that substantially affect performance and translate them into natural-language mutation instructions that steer subsequent LLM-based code generation without restricting expressivity. We propose LLaMEA-SAGE, which integrates this feature-driven guidance into LLaMEA, and evaluate it across several benchmarks. We show that the proposed structured guidance achieves the same performance faster than vanilla LLaMEA in a small controlled experiment. In a larger-scale experiment using the MA-BBOB suite from the GECCO-MA-BBOB competition, our guided approach achieves superior performance compared to state-of-the-art AAD methods. These results demonstrate that signals derived from code can effectively bias LLM-driven algorithm evolution, bridging the gap between code structure and human-understandable performance feedback in automated algorithm design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。