用大模型模拟环境反馈，实现无需真实环境的智能体训练。
Simulating Environments with Reasoning Models for Agent Training
- 用大模型生成多样化任务轨迹，构建无环境依赖的训练数据。
- 在τ²-Bench上超越GPT-4o，接近o4-mini性能。
- 适合想低成本训练鲁棒智能体的研究者与开发者。
大型语言模型（LLM）智能体在需要深度推理的小型环境中表现优异，但在更复杂、多样化的上下文中易出现脆弱性，且依赖特定工具与模式。传统定制化环境训练成本高、难复现，限制了进展。本文证明，无需真实测试平台或API，大模型可模拟真实环境反馈。基于此，提出两个框架：Simia-SFT通过放大少量种子数据生成多样化轨迹，实现环境无关的SFT数据合成；Simia-RL则利用大模型模拟反馈，支持强化学习训练而无需真实环境实现。微调开源模型在多个基准上均取得稳定提升，在τ²-Bench上超越GPT-4o，接近o4-mini表现。二者共同实现无需环境工程的可扩展智能体训练，以灵活的模型仿真替代繁重、脆弱的传统实现。
原文摘要 · Abstract (English)
LLM agents excel in compact environments requiring deep reasoning but remain brittle when operating in broader, more complex contexts that demand robustness across diverse tools and schemas. Building bespoke environments for training is heavy, brittle, and limits progress. In this paper, we demonstrate that LLMs can simulate realistic environment feedback without access to actual testbed data or APIs. Inspired by this capability, we propose two frameworks: Simia-SFT, a pipeline that synthesizes SFT data by amplifying small seed sets into diverse trajectories in an environment-agnostic manner, and Simia-RL, a framework that enables RL training without real environment implementations through LLM-simulated feedback. Fine-tuning open models yields consistent improvements across multiple benchmarks, surpassing GPT-4o and approaching o4-mini on $τ^2$-Bench. Together, Simia-SFT and Simia-RL enable scalable agent training without environment engineering, replacing heavy and brittle implementations with flexible LLM-based simulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。