arXiv:2410.07002cs.CLcs.AI2024-10ICML被引 5

让编程助手更懂上下文,自动整合代码、历史和指令

CursorCore: Assist Programming through Aligning Anything

  • 构建统一框架,融合代码、历史与用户指令
  • 生成21.9万条训练数据,提升模型对多源信息的对齐能力
  • 适用于需要智能代码编辑与对话的开发者

大型语言模型已在代码补全、插入和指令式编辑等编程辅助任务中取得成功,但现有应用仍难以自动化,且在编程过程中难以有效整合代码历史、当前代码和用户指令等多种信息。本文提出一种新型对话式框架,全面集成这些信息源,并构建数据集以训练和评估模型。首先,设计新基准 APEval(Assist Programming Eval),系统评估模型在编程辅助任务中的表现与信息对齐质量。其次,开发 Programming-Instruct 数据生成流水线,从 GitHub 及在线评测平台等多样来源合成训练数据,可自动生成编程过程中的各类消息。最终基于该流水线生成 219,000 条样本,微调多个模型,推出 CursorCore 系列。实验表明,CursorCore 在同等规模模型中表现更优。该框架统一支持内联聊天与自动编辑,推动编程助手发展。代码、模型与数据已开源:https://github.com/TechxGenus/CursorCore。

原文摘要 · Abstract (English)

Large language models have been successfully applied to programming assistance tasks, such as code completion, code insertion, and instructional code editing. However, these applications remain insufficiently automated and struggle to effectively integrate various types of information during the programming process, including coding history, current code, and user instructions. In this work, we propose a new conversational framework that comprehensively integrates these information sources, collect data to train our models and evaluate their performance. Firstly, to thoroughly evaluate how well models align with different types of information and the quality of their outputs, we introduce a new benchmark, APEval (Assist Programming Eval), to comprehensively assess the performance of models in programming assistance tasks. Then, for data collection, we develop a data generation pipeline, Programming-Instruct, which synthesizes training data from diverse sources, such as GitHub and online judge platforms. This pipeline can automatically generate various types of messages throughout the programming process. Finally, using this pipeline, we generate 219K samples, fine-tune multiple models, and develop the CursorCore series. We show that CursorCore outperforms other models of comparable size. This framework unifies applications such as inline chat and automated editing, contributes to the advancement of coding assistants. Code, models and data are freely available at https://github.com/TechxGenus/CursorCore.

编程助手多源对齐代码生成大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。