arXiv:2607.25130cs.SEcs.AI2026-07

收集5.36万条开发者真实编辑AI生成代码的数据,提升编程助手的预测能力。

Learning from 53.6K Real-World Developer Edits of AI-Generated Code

论文配图:Learning from 53.6K Real-World Developer Edits of AI-Generated Code
图 1 · 摘自论文原文
  • 基于开发者在IDE中实际编辑AI生成代码的行为构建数据集
  • 31%的编辑轨迹中删除了AI生成内容,多数在15分钟内完成
  • 用该数据微调小模型,性能超越顶尖大模型

AI生成代码存在缺陷,需开发者手动修改或重新提示编程助手。相比仅记录最终成功代码片段的Git提交,开发者在IDE中的实际编辑行为提供了更真实、更细致的修改信息。然而,高质量、真实的代码编辑数据稀缺,导致大语言模型(LLMs)主要依赖公开的Git提交数据(如commits)进行训练。为填补这一空白,我们提出DECODE(Developer Edits of Code Dataset),包含来自1000多名开发者的53.6万条真实世界中对Python、TypeScript和JavaScript生成代码的编辑记录。首先,我们展示DECODE在数据分析上的价值,揭示了开发者何时、为何以及如何修改AI生成代码:大多数编辑在接收生成代码后的15分钟内发生,且31%的编辑轨迹中删除了原始生成内容。其次,我们利用DECODE评估大模型预测代码编辑的能力,发现对开放源码的30亿参数模型在该数据集上微调后,在代码编辑预测任务中的表现显著优于前沿大模型。最后,本文讨论其意义,强调未来编程助手需采用以开发者为中心的机器学习方法。

原文摘要 · Abstract (English)

Imperfections in AI-generated code require that software developers modify the generated code manually, or by re-prompting an AI programming assistant. Manual code edits provide more realistic and granular information on editing behavior than Git commits, which only contain final successful code snippets. Yet, due to a lack of high-quality, realistic code editing data, LLMs are mostly trained on publicly available Git data (e.g., commits). To address this gap, we introduce DECODE (Developer Edits of Code Dataset), a dataset of 53.6K real-world in-IDE code edits of AI-generated code in Python, TypeScript, and JavaScript, sourced from 1K+ developers. First, we demonstrate the utility of DECODE for data analysis, obtaining insights on when, why, and how AI-generated code is edited. We find that most edits occur within the first 15 minutes after accepting an AI completion, resulting in the removal of AI completions in 31% of edit trajectories. Second, we use DECODE to benchmark the ability of LLMs to predict code edits. We find that finetuning on DECODE enables open-source 3B models to perform code edit prediction tasks significantly better than frontier LLMs. We then discuss implications of this work, emphasizing the necessity of developer-centric machine learning approaches for future AI programming assistants.

代码生成开发者数据大模型微调编程助手

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。