无需参数更新，通过分层记忆实现高效通用多模态智能体
Efficiently Enhancing General Agents With Hierarchical-categorical Memory
- 采用分层记忆检索与任务类别学习双模块架构
- 在多个数据集上达到当前最优性能，显著优于基线方法
- 适合需要持续学习且资源受限的通用智能体场景
随着大语言模型（LLMs）展现出卓越能力，利用其构建通用多模态智能体的研究日益增多。然而，现有方法要么依赖大规模多模态数据的端到端训练，计算成本高昂；要么采用工具调用方式，缺乏持续学习和环境适应能力。本文提出EHC，一种无需参数更新即可学习的通用智能体。EHC包含分层记忆检索（HMR）模块和面向任务类别的经验学习（TOEL）模块。HMR模块实现快速相关记忆检索，并可无容量限制地持续存储新信息；TOEL模块通过经验分类和跨类别模式提取，增强对各类任务特征的理解。在多个标准数据集上的大量实验表明，EHC性能超越现有方法，达到当前最优水平，验证了其在处理复杂多模态任务中的有效性。
原文摘要 · Abstract (English)
With large language models (LLMs) demonstrating remarkable capabilities, there has been a surge in research on leveraging LLMs to build general-purpose multi-modal agents. However, existing approaches either rely on computationally expensive end-to-end training using large-scale multi-modal data or adopt tool-use methods that lack the ability to continuously learn and adapt to new environments. In this paper, we introduce EHC, a general agent capable of learning without parameter updates. EHC consists of a Hierarchical Memory Retrieval (HMR) module and a Task-Category Oriented Experience Learning (TOEL) module. The HMR module facilitates rapid retrieval of relevant memories and continuously stores new information without being constrained by memory capacity. The TOEL module enhances the agent's comprehension of various task characteristics by classifying experiences and extracting patterns across different categories. Extensive experiments conducted on multiple standard datasets demonstrate that EHC outperforms existing methods, achieving state-of-the-art performance and underscoring its effectiveness as a general agent for handling complex multi-modal tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。