让大模型代理通过互动自适应新环境,提升泛化能力。
Test-Time Adaptation for LLM Agents via Environment Interaction
- 用轻量适配向量实时对齐环境格式,解决语法误解问题。
- 部署时探索环境因果动态,构建上下文世界模型,提升复杂任务成功率。
- 在网页导航等任务中显著提效,适合需强泛化的智能体应用。
基于大语言模型(LLM)的智能体在面对全新复杂环境(如未见过的网站或新功能集)时难以泛化,根源在于预训练与测试条件的根本不匹配。这主要源于两类失败模式:对环境特有组件(如观察格式)的语法误解,以及对状态转移动态的语义误解,后者仅在测试时显现。为此,我们提出两种利用部署期交互信息的自适应策略。其一为在线语法对齐(SA),通过学习轻量适配向量,偏置模型输出分布以快速匹配环境响应格式;其二为部署时动态建模(DG),采用角色驱动的探索阶段系统探测并学习环境因果动态,形成上下文世界模型。在函数调用与网页导航等多样化智能体基准上评估表明,两种策略均有效且计算开销极小。尤其在动态不可预测的复杂环境中,动力学建模效果显著——例如在WebArena多站点划分数据集上,成功率达2%提升至23%。代码已开源。
原文摘要 · Abstract (English)
Large language model (LLM)-based agents struggle to generalize to novel and complex environments, such as unseen websites or new sets of functions, due to a fundamental mismatch between their pre-training and test-time conditions. This challenge stems from two distinct failure modes: a syntactic misunderstanding of environment-specific components like observation formats, and a semantic misunderstanding of state-transition dynamics, which are only revealed at test time. To address these issues, we propose two distinct strategies for adapting LLM agents by leveraging environment-specific information from interaction that is available during deployment. First, an online syntactic alignment (SA) method parameterizes environmental nuances by learning a lightweight adaptation vector that biases the model's output distribution, enabling rapid alignment with an environment response format. Second, a deployment-time dynamics grounding (DG) method employs a persona-driven exploration phase to systematically probe and learn the environment's causal dynamics before task execution, equipping the agent with an in-context world model. We evaluate these strategies across diverse agentic benchmarks, including function calling and web navigation. Our empirical results show the effectiveness of both strategies across all benchmarks with minimal computational cost. We find that dynamics grounding is particularly effective in complex environments where unpredictable dynamics pose a major obstacle, demonstrating a robust path toward more generalizable and capable LLM-based agents. For example, on the WebArena multi-site split, this method increases the agent's success rate from 2% to 23%. We release our code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。