小模型通过分阶段训练，让对话游戏代理先能玩起来再提分。
First Make It Playable, Then Make It Good: Staged Interaction Learning for Small Dialogue-Game Agents
- 分三步微调：成功轨迹监督、加权回合训练、教师引导修正
- 最终在Playpen上获57.12分ClemScore，比基线高约36分
- 适合想用小模型高效做对话游戏的开发者参考
我们提出Qwen-GuidePlay-2B，一个20亿参数的语言模型，用于对话游戏交互。通过三阶段微调：a) 在Playpen数据集上仅使用成功游戏轨迹进行SFT；b) 加权回合级SFT；c) 教师引导式SFT（教师仅用于格式修正和评估，不生成新动作）。最终模型在公开Playpen验证集上获得57.12 ClemScore和42.68 Statscore。在官方发布挑战结果中，其Playpen ClemScore提升达第二高（约+36分于基线）。研究发现，模仿完整轨迹有助于可玩性，而回合级与教师引导训练则提升决策能力并整体增分。替代方案如重放修复和硬样本挖掘未见效，表明小模型通过精心数据筛选即可高效，无需激进调整。模型与代码已开源，支持复现。
原文摘要 · Abstract (English)
We present Qwen-GuidePlay-2B, a 2B-parameter language model for dialogue-game interaction. We fine-tune Qwen3.5-2B using three steps: a) SFT on only successful game trajectories from Playpen, b) weighted turn-level SFT, and c) teacher-guided SFT. The teacher model (which is a larger model) is only used to fix formatting and evaluate examples, but does not create new gold actions. Our final model scores 57.12 clemscore and 42.68 statscore on the public Playpen validation. In the officially released challenge results, our model obtains the second-highest Playpen clemscore delta among submitted systems (which is approximately +36 over its base model). Our findings suggest that imitating full trajectories helps with playability, while turn-level and teacher-guided training usually improve decision-making and increase the overall score. Alternative procedurally heavy approaches like replay-repair and hard-example mining did not help, which suggests that small models are performant simply by using careful curation strategies rather than aggressive changes. We make available both the model and the code for reproducibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。