让小模型通过分析对错代码，自动优化生成质量。
CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement
- 用成功与失败代码对比迭代优化，替代单纯学习正确示例。
- 仅用500样本使Llama-3-8B在数据科学任务上准确率从28.2%提升至48.6%。
- 适合资源有限但追求高质量代码生成的研究者和开发者。
大型语言模型（LLMs）虽已革新代码生成，但需大量资源且易过度泛化，限制了其特定任务效率。微调小型开源模型是成本更低的替代方案。然而，传统监督方法仅依赖正确示例，忽略了错误尝试中的宝贵信息。我们提出CodeLutra框架，同时利用正确与错误的代码尝试。不依赖单一正确解，而是通过迭代偏好式精炼，比较成功与失败输出，更精准逼近理想结果。该方法在无需大规模数据集或辅助模型的情况下，缩小了与顶尖大模型的性能差距。例如，在一项具有挑战性的数据科学编码任务中，仅使用500个样本，即把Llama-3-8B的准确率从28.2%提升至48.6%，接近GPT-4水平。通过从成功与错误中学习，CodeLutra为高质量代码生成提供了可扩展、高效的新路径，使小型开源模型更具竞争力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have revolutionized code generation but require significant resources and often over-generalize, limiting their task-specific efficiency. Fine-tuning smaller, open-source LLMs provides a cost-effective alternative. However, standard supervised approaches rely only on correct examples, missing valuable insights from failures. We introduce CodeLutra, a framework that leverages both correct and incorrect code attempts. Instead of using only correct solutions, CodeLutra applies iterative preference-based refinement, comparing successful and failed outputs to better approximate desired results. This approach narrows the performance gap with state-of-the-art larger models without requiring massive datasets or auxiliary models. For instance, on a challenging data science coding task, using only 500 samples improved Llama-3-8B's accuracy from 28.2% to 48.6%, approaching GPT-4's level. By learning from both successes and mistakes, CodeLutra provides a scalable and efficient path to high-quality code generation, making smaller open-source models more competitive with leading closed-source alternatives.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。