轻量微调+数据优化，让代码问题修复模型性能突破极限
SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolving
- 用真实与合成数据构建32k高质量任务集，提升训练效果
- 引入错误掩码和难度课程策略，显著提高生成动作质量
- 仅靠微调就能达顶尖水平，适合追求高效部署的开发者
我们提出SWE-Lego，一种专为软件工程问题修复设计的监督微调方案。不同于依赖复杂训练范式的主流方法，SWE-Lego专注于纯微调的潜力挖掘。其核心包含三个模块：1）SWE-Lego数据集，含32,000个高质量任务实例与18,000条验证轨迹，融合真实与合成数据以互补质量与数量；2）改进的微调流程，结合错误掩码与基于难度的课程学习，显著提升动作质量和整体性能。实证表明，仅凭这两项改进，SWE-Lego-Qwen3-8B在SWE-bench Verified上达到42.2%，SWE-Lego-Qwen3-32B达52.6%，超越同规模开源模型。3）在此基础上进一步优化测试时缩放（TTS），利用预训练验证器增强性能——8B模型从42.2%升至49.6%，32B模型从52.6%升至58.8%（TTS@16）。
原文摘要 · Abstract (English)
We present SWE-Lego, a supervised fine-tuning (SFT) recipe designed to achieve state-ofthe-art performance in software engineering (SWE) issue resolving. In contrast to prevalent methods that rely on complex training paradigms (e.g., mid-training, SFT, reinforcement learning, and their combinations), we explore how to push the limits of a lightweight SFT-only approach for SWE tasks. SWE-Lego comprises three core building blocks, with key findings summarized as follows: 1) the SWE-Lego dataset, a collection of 32k highquality task instances and 18k validated trajectories, combining real and synthetic data to complement each other in both quality and quantity; 2) a refined SFT procedure with error masking and a difficulty-based curriculum, which demonstrably improves action quality and overall performance. Empirical results show that with these two building bricks alone,the SFT can push SWE-Lego models to state-of-the-art performance among open-source models of comparable size on SWE-bench Verified: SWE-Lego-Qwen3-8B reaches 42.2%, and SWE-Lego-Qwen3-32B attains 52.6%. 3) We further evaluate and improve test-time scaling (TTS) built upon the SFT foundation. Based on a well-trained verifier, SWE-Lego models can be significantly boosted--for example, 42.2% to 49.6% and 52.6% to 58.8% under TTS@16 for the 8B and 32B models, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。