构建首个面向第一人称视觉的流式情景记忆诊断基准，可精准测试模型长期记忆能力。
EGOSTREAM: A Diagnostic Benchmark for Streaming Episodic Memory in Egocentric Vision
- 设计7维认知维度问题集，结合答案有效窗口（AVW）实现从即时到超长期回忆的可控测试。
- 在2,250个问题上生成8,528次条件化评估，发现不同记忆机制表现差异显著。
- 揭示当前模型普遍无法实时处理（>1秒/帧），最优仅达45%准确率，暴露架构短板。
持续的情景记忆是自主代理在动态真实环境中运行的核心能力，但现有流式视频基准对模型记忆内容和时长的诊断工具有限。我们提出Egostream，一个用于第一人称视觉中流式情景记忆评估的诊断基准。Egostream包含2,250个精心策划的问题，覆盖细节、空间、时间、事件、社会、因果和前瞻记忆七个认知维度。我们引入答案有效窗口（AVW），定义答案在场景演化中保持有效的时长跨度，从而将问题扩展为8,528次基于回忆条件的评估，实现从即时到超长期回忆的受控测试，并区分真实遗忘与自然世界状态变化。通过统一的流式多模态大模型框架，在Qwen3-VL基础上对比多种先进记忆管理机制——滑动窗口、注意力池、键值缓存修剪、合并与卸载。实验表明，相似总体准确率掩盖了截然不同的记忆特性：例如，标记修剪比合并更有效保留细粒度细节和时间结构；量化卸载则显著提升超长期记忆表现。然而所有机制均远低于实时处理（>1秒/帧），最佳方法准确率仅约45%，暴露出当前架构的关键缺陷。Egostream为填补这些差距提供了必要的诊断平台。
原文摘要 · Abstract (English)
Continuous episodic memory is a core capability for autonomous agents operating in dynamic, real-world environments, yet current streaming video benchmarks provide limited tools for diagnosing what models remember and for how long. We introduce Egostream, a diagnostic benchmark for streaming episodic memory evaluation in egocentric vision. \egostream organizes 2,250 curated questions along seven cognitive dimensions: detail, spatial, temporal, event, social, causal, and prospective memory. We introduce the Answer Validity Window (AVW), which specifies the temporal span an answer remains valid as the observed scene evolves. This allows us to expand the questions into 8,528 recall-conditioned evaluations, enabling controlled testing from instant to ultra-long-term recall while separating genuine model forgetting from natural world-state changes. We rigorously establish baseline performance through a unified streaming MLLM framework that compares several state-of-the-art memory-management mechanisms, covering sliding windows, attention sinks, KV-cache pruning, merging, and offloading. Experiments within a unified Qwen3-VL backbone reveal that comparable aggregate accuracies mask starkly different memory profiles. For instance, token pruning preserves fine-grained details and temporal structure significantly better than token merging, while quantized offloading rescues ultra-long-term recall. Ultimately, all mechanisms operate well below real-time (>1s per frame), and top performing methods ceil at about 45% accuracy, exposing critical gaps in current architectures. Egostream provides the diagnostic testbed needed to close these gaps. Project website, news and updates at: https://saroo25.github.io/Egostream/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。