让智能体在嘈杂环境中训练,提升真实场景下的稳定性。
Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments

- 在训练中引入用户和工具的噪声,模拟真实交互复杂性。
- 噪声逐步增强,模型适应后性能显著提升,理想环境表现也更好。
- 适合关注智能体落地应用与鲁棒性提升的研究者。
大型语言模型(LLMs)的进展推动了具备推理、规划和工具使用能力的交互式智能体的广泛应用。尽管在现有基准上表现优异,这些智能体在真实世界部署时仍常出现性能下降,原因在于训练环境过于理想化。本文提出NoisyAgent框架,将真实场景中的两大噪声源——用户噪声(交互模糊性)和工具噪声(执行失败)——显式纳入训练过程。通过修改用户交互模式并模拟工具执行异常,在训练中动态引入扰动,并仅对部分回溯样本施加噪声,同时随模型适应逐步增加噪声强度以稳定训练。大量实验表明,该方法显著提升了智能体在噪声和动态环境中的鲁棒性。分析还发现,经过噪声训练的智能体在理想化基准上也获得性能提升,说明适度暴露于环境噪声能促进更泛化的推理与决策行为。研究强调,建模交互不完美性对缩小训练与真实部署间的差距至关重要。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have facilitated the widespread deployment of LLMs as interactive agents capable of reasoning, planning, and tool use. Despite strong performance on existing benchmarks, such agents often exhibit notable degradation when deployed in real-world settings, where environments are inherently stochastic and imperfect. We argue that this discrepancy arises from a fundamental mismatch between idealized training settings and real-world interaction dynamics, where current paradigms rely on carefully curated task instructions and stable, well-controlled environments. To address this gap, we propose NoisyAgent, an agentic training framework that explicitly incorporates environmental imperfections into the agent learning process. We identify two major sources of interaction noise in real-world scenarios: user noise, which captures ambiguity and variability in user interaction, and tool noise, which reflects failures and anomalies in tool execution. We introduce such perturbations into the training pipeline by modifying user interaction patterns and simulating tool execution results within the training environment. To stabilize training while encouraging agents to handle increasingly challenging imperfections, noise is applied to only a subset of rollouts and progressively increased in difficulty as the model adapts to the current noise level. Extensive experiments demonstrate that our approach consistently improves agent robustness under noisy and dynamic environments. Our analysis reveals that training under noise conditions also yields performance gains on idealized benchmarks, suggesting that controlled exposure to environmental noise promotes more generalizable reasoning and decision-making behaviors. Our findings highlight the importance of modeling interaction imperfections for bridging the gap between agent training and real-world deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。