构建可执行的多轮操作系统代理数据集,提升智能体任务完成能力。
ISE: An Execution-Grounded Recipe for Multi-Turn OS-Agent Trajectories

- 三阶段合成框架:构造意图、模拟交互、真实执行
- 生成2.3万条完整对话轨迹,平均8.12轮用户交互
- 使小模型性能超越大模型,适合训练实用型智能体
训练高效操作系统智能体需要同时包含结构化用户意图、多轮任务委派和真实工具执行的数据,而现有数据集缺乏这些特性。本文提出ISE(意图→模拟→执行)三阶段合成范式:第一阶段通过4维框架(人物+领域+任务+复杂度)构建约5万条结构化意图,去重后得43956个唯一意图,在mpnet-base-v2嵌入上获得61.57的Vendi分数(余弦核,q=1);第二阶段采用角色锁定的用户模拟器,每轮用户输入基于实际执行结果生成,产出23132条完整轨迹,平均8.12轮用户交互,总计68.24轮对话;第三阶段在隔离的实时操作系统环境中执行所有工具调用,生成真实的失败与恢复动态。在ISETrace数据集上微调Qwen3-8B,ClawEval pass@1从19.3提升至37.7,优于零样本GPT-4o及四倍大的Qwen3-32B基座模型。消融实验表明,多轮模拟贡献了主要性能提升。代码与数据集已开源。
原文摘要 · Abstract (English)
Training capable OS agents requires data that simultaneously captures structured user intents, multi-turn task delegation, and grounded tool execution--properties absent from existing datasets. We propose ISE (Intent -> Simulate -> Execute), a three-stage synthesis paradigm that addresses these gaps jointly. Stage 1 constructs roughly 50000 structured intents via a 4D framework (Persona x Domain x Task x Complexity); after deduplication the pool contains 43956 unique intents and attains a Vendi Score of 61.57 over the entire pool on mpnet-base-v2 embeddings (cosine kernel, q=1). Stage 2 drives multi-turn user-agent interaction through a role-locked user simulator that grounds each user turn in actual execution outcomes, producing 23132 complete trajectories averaging 8.12 user turns and 68.24 total dialogue turns. Stage 3 runs every tool call inside a live, isolated OS workspace, generating authentic failure-recovery dynamics instead of simulated responses. Fine-tuning on ISETrace improves ClawEval pass@1 from 19.3 to 37.7 using Qwen3-8B on agent tool-use tasks with a standard protocol. This result outperforms zero-shot GPT-4o and the larger Qwen3-32B base model which is four times bigger. An ablation on Stage 2 proves multi-turn simulation brings a large portion of the performance gain. We release all source code and dataset at https://github.com/Valiere01/ISE-Trace.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。