让游戏智能体学会复用经验，快速解锁新物品
Experience Transfer for Multimodal LLM Agents in Minecraft Game
- 将过往经验拆解为五维知识，显式支持迁移
- 在《我的世界》中任务解锁速度提升1.3至1.7倍
- 适合需要快速适应新任务的多模态智能体研究者
在复杂游戏环境中，多模态大模型智能体需持续复用历史经验以高效解决新任务。本文提出Echo框架，一种面向迁移的内存机制，使智能体能从过往交互中提取可行动知识，而非仅存储静态记录。为显式支持迁移，Echo将可复用知识分解为结构、属性、过程、功能和交互五个维度，帮助智能体识别跨任务共现模式，并判断已有经验在新情境下的适用性。在此基础上，Echo采用上下文类比学习（ICAL）检索相关经验，并通过上下文示例将其适配到未见任务。在《我的世界》中的实验表明，在零样本学习设定下，Echo在物品解锁任务上实现1.3至1.7倍的速度提升；此外，出现“爆发式连锁解锁”现象，即获取可迁移经验后，短时间内迅速解锁多个相似物品。结果表明，经验迁移是提升多模态大模型智能体在复杂交互环境中的效率与适应性的有效路径。
原文摘要 · Abstract (English)
Multimodal LLM agents operating in complex game environments must continually reuse past experience to solve new tasks efficiently. In this work, we propose Echo, a transfer-oriented memory framework that enables agents to derive actionable knowledge from prior interactions rather than treating memory as a passive repository of static records. To make transfer explicit, Echo decomposes reusable knowledge into five dimensions: structure, attribute, process, function, and interaction. This formulation allows the agent to identify recurring patterns shared across different tasks and infer what prior experience remains applicable in new situations. Building on this formulation, Echo leverages In-Context Analogy Learning (ICAL) to retrieve relevant experiences and adapt them to unseen tasks through contextual examples. Experiments in Minecraft show that, under a from-scratch learning setting, Echo achieves a 1.3x to 1.7x speed-up on object-unlocking tasks. Moreover, Echo exhibits a burst-like chain-unlocking phenomenon, rapidly unlocking multiple similar items within a short time interval after acquiring transferable experience. These results suggest that experience transfer is a promising direction for improving the efficiency and adaptability of multimodal LLM agents in complex interactive environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。