用双向生成与静态验证，自动合成高质量编程指令数据
Infinite-Instruct: Synthesizing Scaling Code instruction Data with Bidirectional Synthesis and Static Verification
- 通过逆向构造和知识图谱重构，提升问题逻辑性
- 7B模型性能提升21.70%，32B模型提升36.95%
- 仅用十分之一数据即达Qwen-2.5-Coder-Instruct水平
传统代码指令数据合成方法存在多样性不足、逻辑差的问题。我们提出Infinite-Instruct，一个自动化框架，用于生成高质量的问答对，以增强大语言模型（LLMs）的代码生成能力。该框架聚焦于提升合成问题的内部逻辑和代码质量。首先，'逆向构造'将代码片段转化为多样化的编程问题；随后，通过'回流构造'，利用问题中的关键词构建知识图谱，重构出逻辑更强的编程问题；最后，采用跨语言静态代码分析管道过滤无效样本，确保数据质量。实验表明，在主流代码生成基准上，微调后的模型在7B参数模型上平均性能提升21.70%，在32B参数模型上提升36.95%。使用不到十分之一的指令微调数据，即达到Qwen-2.5-Coder-Instruct的性能水平。Infinite-Instruct为编程领域大模型训练提供了可扩展解决方案。实验所用数据集已开源，包含未过滤和经静态分析过滤的版本，地址为https://github.com/xingwenjing417/Infinite-Instruct-dataset。
原文摘要 · Abstract (English)
Traditional code instruction data synthesis methods suffer from limited diversity and poor logic. We introduce Infinite-Instruct, an automated framework for synthesizing high-quality question-answer pairs, designed to enhance the code generation capabilities of large language models (LLMs). The framework focuses on improving the internal logic of synthesized problems and the quality of synthesized code. First, "Reverse Construction" transforms code snippets into diverse programming problems. Then, through "Backfeeding Construction," keywords in programming problems are structured into a knowledge graph to reconstruct them into programming problems with stronger internal logic. Finally, a cross-lingual static code analysis pipeline filters invalid samples to ensure data quality. Experiments show that on mainstream code generation benchmarks, our fine-tuned models achieve an average performance improvement of 21.70% on 7B-parameter models and 36.95% on 32B-parameter models. Using less than one-tenth of the instruction fine-tuning data, we achieved performance comparable to the Qwen-2.5-Coder-Instruct. Infinite-Instruct provides a scalable solution for LLM training in programming. We open-source the datasets used in the experiments, including both unfiltered versions and filtered versions via static analysis. The data are available at https://github.com/xingwenjing417/Infinite-Instruct-dataset
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。