让AI通过执行反馈自我进化,提升私有库代码生成准确率。
Learning from Execution: Self-Evolving Memory for Private-Library Code Generation

- 构建多层级可演化的记忆系统,从执行结果中提炼使用规范。
- 在两个评测集上平均提升18.41%的代码通过率,显著优于现有方法。
- 适合需要精准调用内部私有库的企业级代码生成场景。
大型语言模型在通用代码生成上表现优异,但在依赖内部私有库的企业场景中效果急剧下降,因这些库未出现在公开预训练语料中。现有检索增强生成(RAG)方法虽无需训练,仅依赖静态API文档,但分析显示文档仅帮助识别应使用哪些API,难以指导正确用法。即使拥有理想文档检索,模型仍频繁出现API误用、组合错误或策略偏差。为此,本文提出MEMCoder——一种无需训练的自演化记忆框架。MEMCoder在现有RAG流程中引入多层级演化记忆,持续积累并复用执行反馈所得的API级、跨API级和任务级使用指南。生成时同时检索文档与历史记忆;执行后分析反馈,通过生成-执行-反思-更新闭环优化记忆。在NdonnxEval和NumbaEval上的实验证明,MEMCoder在不同规模模型和多种RAG架构下均显著提升性能,平均绝对提升pass@1达18.41个百分点。其多层级记忆结构有效性得到验证,且优于现有自演化记忆方法。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have achieved strong performance on general code generation, but their effectiveness drops sharply in enterprise settings where software development relies on internal private libraries absent from public pre-training corpora. Existing Retrieval-Augmented Generation (RAG) methods provide a training-free solution by retrieving static API documentation, but our analysis shows that documentation mainly helps models identify what APIs to use and remains insufficient for teaching how to use them correctly. Even with oracle API-document retrieval, LLMs still make recurring errors at the API, cross-API, and task levels, including API misuse or hallucination, flawed API composition, and incorrect solution strategies. To address this limitation, we propose MEMCoder, a training-free self-evolving memory framework for private-library code generation. MEMCoder augments existing RAG pipelines with a Multi-level Evolving Memory that continuously accumulates and reuses execution-derived Usage Guidelines at the API, cross-API, and task levels. During generation, MEMCoder retrieves both static API documentation and relevant historical memories to guide code generation; after execution, it analyzes feedback to refine memory through a closed loop of generation, execution, reflection, and update. Extensive experiments on NdonnxEval and NumbaEval show that MEMCoder consistently enhances different RAG backbones across LLMs of different scales, yielding an average absolute pass@1 improvement of 18.41 percentage points. Moreover, MEMCoder outperforms existing self-evolving memory methods and validates the effectiveness of organizing execution feedback into multi-level usage memories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。