用在线强化学习让小模型在本地自动配置开发环境
PIPer: On-Device Environment Setup via Online Reinforcement Learning
- 结合监督微调与可验证奖励的强化学习，优化环境配置脚本生成
- Qwen3-8B在环境配置任务上达到与32B大模型相当效果
- 适合需要在本地自动化部署开发环境的研究者和开发者
环境配置——即为特定软件项目设置系统运行环境——是软件工程中持续存在的挑战。自动化环境配置方法可帮助开发者无需手动操作即可为任意代码仓库提供完整配置环境，同时助力软件工程研究人员扩展基于执行的基准测试规模。然而，近期研究表明，即使是最先进的大语言模型在该任务上也表现有限。为此，我们针对环境配置任务训练了一个专用模型：结合监督微调生成正确的Bash脚本，并采用可验证奖励的强化学习（RLVR）进行适应性优化。在EnvBench-Python数据集上，我们的方法使可在消费级硬件运行的Qwen3-8B模型在性能上达到与更大模型Qwen3-32B和GPT-4o相当水平。训练代码与模型检查点已公开：https://github.com/JetBrains-Research/PIPer。
原文摘要 · Abstract (English)
Environment setup-the process of configuring the system to work with a specific software project-represents a persistent challenge in Software Engineering (SE). Automated environment setup methods could assist developers by providing fully configured environments for arbitrary repositories without manual effort. This also helps SE researchers to scale execution-based benchmarks. However, recent studies reveal that even state-of-the-art Large Language Models (LLMs) achieve limited success in automating this task. To address this limitation, we tune a specialized model for environment setup. We combine supervised fine-tuning for generating correct Bash scripts and Reinforcement Learning with Verifiable Rewards (RLVR) to adapt it to the task of environment setup. On EnvBench-Python, our method enables Qwen3-8B (a model runnable on consumer hardware) to perform on par with larger models-Qwen3-32B and GPT-4o. The training code and model checkpoints are available online: https://github.com/JetBrains-Research/PIPer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。