评测多用户共享记忆的智能体,发现现有方法难以兼顾实用、安全与删除可靠性。
GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agents

- 设计跨角色、跨场景的多主共享记忆评估框架
- 多数模型在权限控制和删除遗忘上表现不佳,存在信息泄露
- 适合研究多用户AI系统安全与记忆治理的学者和工程师
大语言模型智能体的记忆评估大多基于单用户设定,而医院、办公室、校园和家庭等共享场景中,多个主体共同使用同一记忆池,各自以不同角色、范围和关系写入与查询,因此记忆质量不仅依赖召回,更需治理。我们提出GateMem,一个面向多主共享记忆智能体的基准测试。它联合评估长期请求的实用性、状态更新能力、上下文授权边界下的访问控制,以及显式删除后的主动遗忘。涵盖医疗、办公、教育和家庭领域,包含长篇多角色对话、增量记忆注入、隐藏检查点、结构化评判与泄露目标标注。在多种基线和骨干模型上,无一方法能同时实现高实用性、强访问控制和可靠遗忘。长上下文提示虽性能最佳但成本高昂,基于检索和外部记忆的方法虽降低成本仍存在未授权或已删除信息泄露。结果表明当前记忆智能体距离可靠的机构级共享部署仍有显著差距。
原文摘要 · Abstract (English)
Memory benchmarks for LLM agents largely assume single-user settings, leaving shared assistants for hospitals, workplaces, campuses, and households understudied. In these deployments, multiple principals write to a common memory pool and query it under different roles, scopes, and relationships, so memory quality requires governance as well as recall. We introduce GateMem, a benchmark for multi-principal shared-memory agents. GateMem jointly evaluates utility for legitimate long-horizon requests with state updates, access control across contextual authorization boundaries, and agent-facing active forgetting after explicit deletion requests. It spans medical, office, education, and household domains, with long-form multi-party episodes, incremental memory injection, hidden checkpoints, structured judging, and leak-target annotations. Across diverse baselines and backbone models, no method simultaneously achieves strong utility, robust access control, and reliable forgetting. Long-context prompting often yields the best governance score at high token cost, while retrieval-based and external-memory methods reduce cost yet still leak unauthorized or deleted information. These results show current memory agents remain far from reliable shared institutional deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。