根据智能体表现动态生成难度适配的训练环境，提升强化学习效果。
Envs-FORGE: Frontier-Optimized Reward-Grounded Environment Synthesis for Agent RL
- 基于验证奖励反馈，为每个种子动态设计任务环境。
- 在tb-core数据集上提升通过率9.2个百分点至49.2%。
- 适用于大模型强化学习训练，适合研究高效环境合成方法者。
终端智能体的强化学习需要可执行、奖励可靠且难度适中的训练环境。现有固定提示策略（如few-shot、Self-Instruct、Evol-Instruct）对所有种子采用相同指令，无法根据当前策略需求调整任务难易度。本文提出Envs-FORGE，一种将验证器奖励转化为每种子环境生成动作的提示策略。该方法估计种子通过率，评估六个投影方向动作以逼近目标学习前沿，并求解每个种子的混合整数线性规划（MILP），选择最优生成动作。该动作同步修改指令、测试用例、标准解和Docker环境；仅通过验证的环境包进入训练。索引化的MILP形式还支持可选的软技能覆盖率用于组合规划。在Qwen 3.5 35B上，Envs-FORGE使tb-core的Pass@1提升9.2个百分点（40.0% → 49.2%），tb-2.0提升6.4点（23.0% → 29.4%），超过最强固定配方基线2.4和2.1点。在SWE-bench Verified上达77.1%，优于基线73.4%。在4B–35B各模型上，tb-core提升6.8–9.2点。所有合成方法均生成100个已验证环境，使用2.27M–2.88M合成标记，确保下游训练集规模与操作尺度一致。源码见：https://github.com/DataArcTech/DataArc-SynData-Toolkit/
原文摘要 · Abstract (English)
Reinforcement learning (RL) for terminal agents needs executable training environments with reliable rewards and useful difficulty. Fixed recipes such as few-shot, Self-Instruct, and Evol-Instruct apply the same prompting policy to every seed, even when the current policy would benefit from a harder, easier, or simply different task. We present Envs-FORGE, a prompting policy that converts verifier rewards into per-seed environment-synthesis actions. Envs-FORGE estimates seed pass rates, scores six projection--direction actions around a target learning frontier, and solves a per-seed mixed-integer linear program (MILP) to choose the action that conditions generation. The selected action drives synchronized rewriting of the instruction, fixtures, oracle solution, tests, and Docker environment; only gold-verified bundles enter RL training. The indexed MILP form also supports optional soft skill coverage for portfolio planning. On Qwen 3.5 35B, Envs-FORGE improves Pass@1 over Base by 9.2 percentage points on tb-core (40.0% to 49.2%) and 6.4 points on tb-2.0 (23.0% to 29.4%), exceeding the strongest fixed-recipe baseline by 2.4 and 2.1 points. It reaches 77.1% on SWE-bench Verified versus 73.4% for Base, and improves tb-core by 6.8--9.2 points across the evaluated 4B--35B models. All synthesis methods export 100 verified environments and use 2.27M--2.88M synthesis tokens, placing the comparison at the same downstream training-set size and the same operational scale. The source code is available at https://github.com/DataArcTech/DataArc-SynData-Toolkit/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。