arXiv:2509.03956cs.AI2025-09ICML被引 7

让智能体在测试时动态接入世界模型,实现零样本适应新环境。

World Model Implanting for Test-time Adaptation of Embodied Agents

  • 用原型检索匹配轨迹抽象表示,动态植入领域特定世界模型。
  • 跨域适应表现优于多个LLM基线,在未见环境中实现零样本与少样本学习。
  • 适合需要高效适应新场景的机器人或虚拟代理系统使用。

在具身人工智能中,如何让智能体在不依赖大量数据收集或重训练的情况下,稳健适应新领域仍是一大挑战。为此,我们提出世界模型植入框架(WorMI),通过测试时组合大型语言模型(LLMs)的推理能力与独立学习的领域特定世界模型,实现智能体策略的跨域自适应。该框架采用基于原型的世界模型检索方法,利用高效的基于轨迹的抽象表示匹配,实现相关模型的无缝植入与移除。同时,设计了一种全局复合注意力机制,不仅融合所检索世界模型的知识,还对齐其中间表示与推理模型在智能体策略中的表征。该设计有效整合多个世界模型的领域知识,确保对未见领域的鲁棒适应。我们在VirtualHome和ALFWorld基准上评估了WorMI,结果表明其在多种未见领域中均显著优于多个基于LLM的方法,展现出卓越的零样本与少样本性能。这凸显了该框架在需高适应性与数据效率的具身智能实际部署中的潜力。

原文摘要 · Abstract (English)

In embodied AI, a persistent challenge is enabling agents to robustly adapt to novel domains without requiring extensive data collection or retraining. To address this, we present a world model implanting framework (WorMI) that combines the reasoning capabilities of large language models (LLMs) with independently learned, domain-specific world models through test-time composition. By allowing seamless implantation and removal of the world models, the embodied agent's policy achieves and maintains cross-domain adaptability. In the WorMI framework, we employ a prototype-based world model retrieval approach, utilizing efficient trajectory-based abstract representation matching, to incorporate relevant models into test-time composition. We also develop a world-wise compound attention method that not only integrates the knowledge from the retrieved world models but also aligns their intermediate representations with the reasoning model's representation within the agent's policy. This framework design effectively fuses domain-specific knowledge from multiple world models, ensuring robust adaptation to unseen domains. We evaluate our WorMI on the VirtualHome and ALFWorld benchmarks, demonstrating superior zero-shot and few-shot performance compared to several LLM-based approaches across a range of unseen domains. These results highlight the frameworks potential for scalable, real-world deployment in embodied agent scenarios where adaptability and data efficiency are essential.

具身智能世界模型测试时适应LLM融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。