评测智能体如何利用实时观察和交互记忆，为未来任务提供帮助。
StreamMemBench: Streaming Evaluation of Agent Memory for Future-Oriented Assistance
- 构建两阶段任务流程，测试从观察到未来任务的连续记忆使用。
- 8个系统在证据复用上表现差，反馈未有效转化为后续行为。
- 适合研究长期记忆、个性化助手的学者与开发者参考。
个人智能体记忆的核心作用是将存储信息和过往交互转化为面向未来的辅助能力。日常使用中，有用线索来自智能体的观察和用户交互，这些信息需从当前请求延续至后续相似任务。现有记忆评测多孤立测试对话回忆或任务提升，未能覆盖从持续观测到未来辅助的完整路径。我们提出 StreamMemBench，基于 EgoLife 的第一人称视频流，为每个证据锚点构建两阶段任务序列：第一阶段测试证据使用，第二阶段检验反馈与交互经验是否被重用。四个指标分别诊断证据回忆、初始使用、反馈融合与后续复用能力。在两种主干模型上对八个记忆系统实验发现，即使证据已存或反馈局部整合，当前系统仍普遍无法有效利用观察信息或将反馈转化为可靠后续行为。StreamMemBench 已开源：https://github.com/landian60/StreamMemBench。
原文摘要 · Abstract (English)
A central role of personal-agent memory is to turn stored information and prior interactions into future-oriented assistance. In daily use, useful cues come from what the agent observes and how the user interacts with the agent, and the agent must carry them forward from the current request to similar future tasks. Existing memory benchmarks usually test dialogue recall or task improvement in isolation, leaving the trajectory from streaming observations to later assistance largely untested. We introduce StreamMemBench, a streaming benchmark that constructs a two-step task sequence around each evidence anchor from EgoLife egocentric streams. The initial task tests evidence use, while the follow-up task tests whether feedback and interaction experience are reused. Four metrics diagnose evidence recall, initial evidence use, feedback incorporation, and follow-up reuse. Experiments with eight memory systems across two backbones show that current systems often fail to use observed evidence or turn feedback into reliable follow-up behavior, even when evidence is stored or feedback is incorporated locally. StreamMemBench is publicly available at https://github.com/landian60/StreamMemBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。