用开源大模型生成高质量代码编辑数据，提升模型性能。
Generating High-Quality Datasets for Code Editing via Open-Source Language Models
- 用多模型协作合成真实代码编辑三元组，包含简洁与详细指令。
- 构建2万样本数据集，使模型在基准上提升4.5%至20.79%。
- 无需人工标注或闭源资源，逼近GPT-4性能，适合代码生成研究者。
代码编辑在软件工程中至关重要，需根据自然语言指令调整代码并保持功能完整、避免冗余修改。但现有基于提交的数据库常含噪声、多样性不足，且无法反映真实编辑指令风格。为此，我们提出OpenCodeEdit——一个开源管道，利用多个大语言模型合成逼真的代码编辑三元组。该流程生成简洁的“懒惰式”和详尽的“描述式”指令，并通过差异（diff）和主题过滤保证数据质量与多样性。基于此，我们构建了2万样本的精选数据集OCEDataFT。在该数据集上微调三个先进基础模型，在CanItEdit基准测试中实现4.50%至20.79%的相对pass@1提升。显著的是，生成模型性能接近闭源系统，与GPT-4差距缩小至3.54%，且不依赖专有资源或人工标注。
原文摘要 · Abstract (English)
Code editing plays a vital role in software engineering, requiring developers to adjust existing code according to natural language instructions while keeping functionality intact and avoiding unnecessary modifications. However, commit-based datasets commonly used for this task are often noisy, lack diversity, and fail to reflect the style of real-world edit instructions. To address this, we introduce OpenCodeEdit, an open-source pipeline that leverages multiple LLMs to synthesize realistic code-edit triplets. The pipeline produces both concise "lazy" instructions and more detailed "descriptive" ones, and applies filtering based on diffs and topics to guarantee data quality and variety. Using this process, we construct OCEDataFT, a curated dataset of 20K samples. Fine-tuning three advanced base models on OCEDataFT leads to significant performance boosts on the CanItEdit benchmark, with relative pass@1 improvements ranging from 4.50% to 20.79%. Notably, the resulting models achieve performance close to closed-source systems, narrowing the gap to GPT-4 to just 3.54%, without relying on proprietary resources or manual annotation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。