arXiv:2608.19880cs.AIcs.CL2026-08

让静态环境自动变难,随智能体成长动态调整挑战。

EnvHarness: Awakening Static Worlds for Agent Learning

论文配图:EnvHarness: Awakening Static Worlds for Agent Learning
图 1 · 摘自论文原文
  • 用可插拔组件层改造静态环境行为,不改底层逻辑。
  • 在5个基准上提升9.0分,减少9.8%执行步数。
  • 适合强化学习研究者,支持智能体与环境共进化。

LLM智能体通过与环境交互学习,但现有环境多为人工构建且静态不变,无法感知智能体弱点,也难以随其进步而更新。尽管近期有环境生成方法尝试解决此问题,但需领域专用流程,依赖昂贵或不可靠的验证器,且仍生成静态环境。为减轻从头重建环境的工程负担,我们提出环境抓手(EnvHarness),一个可编程的插件式组件层,可包裹静态环境并重塑其行为,无需修改底层逻辑。通过标准接口运行,跨多个领域通用,并确保每个重构环境保留原有验证器。为实现自动化,我们引入EnvRigger,将目标策略视为黑箱,通过观察其执行轨迹诊断缺陷,合成针对性的EnvHarness组件,并通过新采样进行验证。在四个领域的五个基准测试中,EnvHarness优于原始环境及领域专用环境生成流程,在保留实例上最高提升9.0分,执行步数减少9.8%。此外,EnvHarness为强化学习提供更优优化信号,支持策略与环境的持续、定向共演化。

原文摘要 · Abstract (English)

LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.

强化学习环境生成共演化智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。