构建可扩展智能体强化学习仿真环境，解决静态评估不足问题
Beyond Static Evaluation: Building Simulation Environments for Scalable Agentic Reinforcement Learning
- 设计API与UI驱动的RL环境，分离环境创建与执行
- 实现高保真行为轨迹与多维奖励塑造，提升训练效果
- 适合研究智能体自主决策与模型优化的开发者
随着大语言模型演变为自主智能体，传统静态评估无法捕捉多步决策过程。我们提出AgenticAI-Supervisor，一个基于API与UI的强化学习环境，将环境构建与可扩展执行解耦。通过可验证的执行结果，平台生成高保真行为轨迹，并应用多维度奖励塑造。关键在于，框架通过严格的内部状态验证与测试，有效缓解奖励劫持问题。本文通过客户支持智能体案例，展示了模型优化所需的闭环反馈机制。未来工作将聚焦于计算机使用、工具调用、自动‘难例’生成及边缘情况构造等高级功能。
原文摘要 · Abstract (English)
As Large Language Models (LLMs) evolve into autonomous agents, traditional static evaluation fails to capture multi-step decision-making. We introduce AgenticAI-Supervisor, an API and UI-driven RL Gym environment that decouples environment creation from scalable execution. By moving to verifiable execution outcomes, the platform generates high-fidelity traces and applies multi-dimensional reward shaping. Critically, our framework mitigates reward hacking through rigorous internal state validation and testing. This work provides a first look at our platform's core capabilities through a Customer Support Agent case study demonstrating a consistent closed-loop feedback for model optimization. Future work will focus on advanced features such as Computer Use, Tool Use, automated "stumping", and edge-case generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。