arXiv:2601.18418cs.SEcs.AI2026-01被引 13

用真实开发流程数据训练代码智能体,提升自动化编程能力。

daVinci-Dev: Agent-native Mid-training for Software Engineering

  • 构建两类原生代理数据:上下文完整与环境真实交互轨迹。
  • 32B和72B模型在SWE-Bench上分别达到56.1%和58.5%解决率。
  • 仅用不足一半的训练词元(731亿),优于现有开源方案。

大型语言模型在软件工程中的前沿已从单轮代码生成转向智能体式开发——即模型能自主导航、编辑并测试复杂代码库。尽管后训练方法已成为代码智能体的主流,但基于大规模真实开发流程数据的中段训练(mid-training)仍因资源消耗大而未被充分探索,其潜力在于比昂贵的强化学习更可扩展地培养基础智能体行为。核心挑战在于静态训练数据与动态、反馈密集的真实开发环境之间的分布差异。为此,本文系统研究了智能体中段训练,提出数据合成原则与训练方法。核心是‘代理原生数据’,包含两类互补轨迹:上下文原生轨迹保留智能体完整信息流,覆盖广、多样性高;环境原生轨迹来自可执行仓库,观察源自实际工具调用与测试执行,具备深度与交互真实性。我们在`SWE-Bench Verified`上验证模型智能体能力,结果表明,在使用对齐基座模型和智能体架构的两种后训练设置下,我们的方法优于此前开源的`Kimi-Dev`方案,且训练词元不足其一半(73.1B)。此外,最佳32B和72B模型分别达到56.1%和58.5%的解决率。

原文摘要 · Abstract (English)

Recently, the frontier of Large Language Model (LLM) capabilities has shifted from single-turn code generation to agentic software engineering-a paradigm where models autonomously navigate, edit, and test complex repositories. While post-training methods have become the de facto approach for code agents, **agentic mid-training**-mid-training (MT) on large-scale data that mirrors authentic agentic workflows-remains critically underexplored due to substantial resource requirements, despite offering a more scalable path to instilling foundational agentic behaviors than relying solely on expensive reinforcement learning. A central challenge in realizing effective agentic mid-training is the distribution mismatch between static training data and the dynamic, feedback-rich environment of real development. To address this, we present a systematic study of agentic mid-training, establishing both the data synthesis principles and training methodology for effective agent development at scale. Central to our approach is **agent-native data**-supervision comprising two complementary types of trajectories: **contextually-native trajectories** that preserve the complete information flow an agent experiences, offering broad coverage and diversity; and **environmentally-native trajectories** collected from executable repositories where observations stem from actual tool invocations and test executions, providing depth and interaction authenticity. We verify the model's agentic capabilities on `SWE-Bench Verified`. We demonstrate our superiority over the previous open software engineering mid-training recipe `Kimi-Dev` under two post-training settings with an aligned base model and agentic scaffold, while using less than half mid-training tokens (73.1B). Besides relative advantage, our best performing 32B and 72B models achieve **56.1%** and **58.5%** resolution rates, respectively, which are ...

智能体代码生成中段训练软件工程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。