用程序能否正常运行来训练代码生成模型,显著提升跨游戏类型生成能力。
The Verifier is the Curriculum: Execution-Gated Self-Distillation for Cross-Family Game Generation
- 以程序能否无头运行作为筛选标准,驱动自蒸馏优化生成质量。
- 跨四类未见游戏类型,生成成功率从8.8%提升至42.2%,全覆盖率达100%。
- 验证器的严格性是关键,比单纯增加数据更有效,适合做生成模型训练。
在代码生成模型后训练中,使用学习到的评判器会优化代理特征但未必提升实际产出。本文研究反向信号:一种确定性、无需评判、不可作弊的过滤机制——生成项目能否在无头引擎下顺利启动(严格启动)。在此筛选条件下,拒绝采样自蒸馏显著提升了跨家族泛化能力。在GameCraft-Bench(将自然语言描述映射为完整Godot项目)上,一个14B模型(Qwen3-14B+LoRA)经过三轮严格启动筛选的自蒸馏,使四类未见游戏家族的清洁生成率从8.8%提升至42.2%(每候选),最佳覆盖率达25/25(黄金上限),各轮提升均显著(p=0.0019, p<1e-4, p<1e-4)。该增益非仅因数据增加:与完全匹配的黄金复制对照组相比,其性能反而下降(5.6% vs. 8.8%,p=0.019);而数量匹配分解显示,第一轮到第二轮的提升可拆分为质量(+8.8pp)与数量(+8.5pp)两通道。直接验证:若将严格启动门换为宽松的BUILD检查(通过率达99.9%),则增益完全消失(回归基线,p=1e-3 vs. 启动门轮次),证明验证器精度才是核心。第二个不可作弊信号——无头执行接地,在各轮持续上升,且在相同预算下生成的可靠候选数远超黄金复制(16 vs. 5),确认收益来自功能性提升而非空壳启动。游戏生成是一个可验证的测试平台,揭示一条核心经验:验证器即课程——它所认证的内容,就是模型真正学会的东西。
原文摘要 · Abstract (English)
Post-training a code generator against a learned judge can optimize proxy features that raise the score without improving the artifact. We study the opposite signal: a deterministic, judge-free, ungameable filter -- whether a generated project launches cleanly under a headless engine (strict-launch). Under this gate, rejection-sampling self-distillation compounds out-of-family generalization. On GameCraft-Bench (mapping a natural-language brief to a complete Godot project), a 14B model (Qwen3-14B+LoRA) distilled under strict-launch raises clean generation on four unseen game families from 8.8% to 42.2% per-candidate and best-of-K coverage from 18/25 to 25/25 (the gold ceiling) over three rounds, each a significant gain (p=0.0019, p<1e-4, p<1e-4). The gain is not from merely adding data: an exactly-matched gold-duplication control regresses below the base model (5.6% vs. 8.8%, p=0.019), while a count-matched decomposition splits the round-1-to-2 jump into comparable quality (+8.8pp) and quantity (+8.5pp) channels. Most directly, rerunning the loop with only the filter swapped -- the lenient BUILD check, which passes 99.9% of generations, in place of the launch gate -- erases the gain entirely (back to base, p=1e-3 vs. the launch-gated round), isolating verifier precision rather than the optimizer. A second ungameable signal, headless execution grounding, rises monotonically across rounds and yields far more grounded candidates than gold-duplication at a matched budget (16 vs. 5), confirming the gains are functional, not launch-but-empty. Game generation is a verifiable testbed for one lesson: the verifier is the curriculum -- what it certifies is what the model learns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。