通过图谱分析LLM生成代码的演化过程，揭示其设计规律与性能瓶颈。
Code Evolution Graphs: Understanding Large Language Model Driven Design of Algorithms
- 构建代码演化图谱，追踪LLM在迭代提示中生成代码的变化路径。
- 重复提示使代码复杂度上升，但过度复杂会降低算法性能。
- 不同LLM有独特编程风格，多模型协同可提升代码质量。
大型语言模型（LLMs）在代码生成方面展现出巨大潜力，尤其在进化计算框架中通过反复迭代优化生成算法。然而，在某些情况下，模型无法生成具有竞争力的算法或优化过程停滞，且由于缺乏对生成过程和代码演化的理解，难以进行干预。本文提出一种新方法，通过构建代码演化图谱，帮助用户分析在反复提示下生成代码的演变过程及其内在规律。我们在三个基准问题类别上进行了验证，获得新发现：重复提示会使生成的代码趋于更复杂，但额外复杂性在某些情况下反而损害算法性能；不同LLM具有不同的编码风格，生成代码彼此差异显著。这些发现表明，在代码演化框架中使用多种LLM，可能比单一模型产生更优的代码。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated great promise in generating code, especially when used inside an evolutionary computation framework to iteratively optimize the generated algorithms. However, in some cases they fail to generate competitive algorithms or the code optimization stalls, and we are left with no recourse because of a lack of understanding of the generation process and generated codes. We present a novel approach to mitigate this problem by enabling users to analyze the generated codes inside the evolutionary process and how they evolve over repeated prompting of the LLM. We show results for three benchmark problem classes and demonstrate novel insights. In particular, LLMs tend to generate more complex code with repeated prompting, but additional complexity can hurt algorithmic performance in some cases. Different LLMs have different coding ``styles'' and generated code tends to be dissimilar to other LLMs. These two findings suggest that using different LLMs inside the code evolution frameworks might produce higher performing code than using only one LLM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。