arXiv:2604.01007cs.AI2026-04被引 11

用自动研究发现终身多模态记忆框架,性能提升超4倍。

Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory

  • 通过自主实验搜索,发现高效多模态记忆架构
  • 在两个数据集上F1分别提升411%和214%
  • 关键突破来自修复漏洞和结构改进,非调参

AI智能体需长期运行,但其对多模态经验的存储、组织与召回能力仍是瓶颈。构建有效终身记忆需探索包含架构、检索策略、提示工程和数据流水线的庞大设计空间,传统方法难以覆盖。本文部署自主研究流程,发现Omni-SimpleMem:一个统一的多模态终身记忆框架。从基础配置(LoCoMo上F1=0.117)出发,自主执行约50次实验,诊断失败模式、提出架构修改并修复数据管道错误,全程无需人工干预。最终系统在两个基准上均达领先水平,LoCoMo上F1提升至0.598(+411%),Mem-Gallery上从0.254升至0.797(+214%)。最关键的是,最显著改进来自漏洞修复(+175%)、架构变更(+44%)和提示工程(特定类别+188%),均超过所有超参数调优的总贡献,证明超越传统AutoML的能力。本文归纳六类发现类型,识别四条使多模态记忆适合自主研究的特性,为其他系统提供指导。代码已开源。

原文摘要 · Abstract (English)

AI agents increasingly operate over extended time horizons, yet their ability to retain, organize, and recall multimodal experiences remains a critical bottleneck. Building effective lifelong memory requires navigating a vast design space spanning architecture, retrieval strategies, prompt engineering, and data pipelines; this space is too large and interconnected for manual exploration or traditional AutoML to explore effectively. We deploy an autonomous research pipeline to discover Omni-SimpleMem, a unified multimodal memory framework for lifelong AI agents. Starting from a naïve baseline (F1=0.117 on LoCoMo), the pipeline autonomously executes ${\sim}50$ experiments across two benchmarks, diagnosing failure modes, proposing architectural modifications, and repairing data pipeline bugs, all without human intervention in the inner loop. The resulting system achieves state-of-the-art on both benchmarks, improving F1 by +411% on LoCoMo (0.117$\to$0.598) and +214% on Mem-Gallery (0.254$\to$0.797) relative to the initial configurations. Critically, the most impactful discoveries are not hyperparameter adjustments: bug fixes (+175%), architectural changes (+44%), and prompt engineering (+188% on specific categories) each individually exceed the cumulative contribution of all hyperparameter tuning, demonstrating capabilities fundamentally beyond the reach of traditional AutoML. We provide a taxonomy of six discovery types and identify four properties that make multimodal memory particularly suited for autoresearch, offering guidance for applying autonomous research pipelines to other AI system domains. Code is available at this https://github.com/aiming-lab/SimpleMem.

终身记忆自动研究多模态智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。