arXiv:2608.02392cs.CVcs.AI2026-08

让可穿戴设备从视频流中自动构建分层记忆,既能回答问题也能主动提醒。

GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience

论文配图:GROVE: Growing and Reasoning over Temporally Stratified Memory from Streaming Video Experience
图 1 · 摘自论文原文
  • 从连续视频流中按时间分层构建记忆,保留细节并归纳成片段、事件和跨日规律。
  • 在MM-lifelong和EgoServe两个挑战性数据集上表现最佳,跨日规律记忆提升显著。
  • 适合需要长期视觉记忆的智能助手,无需训练即可实现问答与主动服务统一记忆

可穿戴助手应能回答关于视觉历史的问题,并识别历史何时对当前情境有用。现有视频记忆系统主要支持基于问题的回忆,而主动型助手通常采用独立的记忆与控制机制。我们提出GROVE,一种无需训练的框架,通过从连续视频流中因果式生成单一记忆,同时支持这两种行为。GROVE保留细粒度感知证据,并逐步整合为时间标记的时刻、连贯的事件以及重复出现的跨日模式。每一层次均配备相应的原生检索技能,用于定位观察、重放活动或遍历长周期规律。反应式问答与主动协助共享同一记忆与访问接口,区别仅在于检索是否由用户提问或当前情境触发。在多个基准测试(包括具有挑战性的MM-lifelong和EgoServe)中,GROVE优于对比方法。受控消融实验表明,时间分层及其检索技能具有互补性,当证据跨越多天时,模式记忆带来的收益最大。代码将发布于https://github.com/SitongGong/GROVE。

原文摘要 · Abstract (English)

A wearable assistant should both answer questions about its visual history and recognize when that history is useful to the present situation. Existing video-memory systems primarily support question-conditioned recall, whereas proactive assistants typically use separate memory and control mechanisms. We introduce GROVE, a training-free framework that supports both behaviors with one memory grown causally from a continuous video stream. GROVE retains fine-grained perceptual evidence and incrementally consolidates it into time-stamped moments, coherent episodes, and recurring cross-day patterns. Each stratum is paired with a scale-native retrieval skill for locating an observation, replaying an activity, or traversing long-range regularities. Reactive QA and proactive assistance share this memory and access interface, differing in whether retrieval is initiated by a user query or the current situation. Across multiple benchmarks including the challenging MM-lifelong and EgoServe, GROVE achieves the best results among the compared methods. Controlled ablations show that the temporal strata and their access skills are complementary, with patterns providing the largest benefit when evidence spans multiple days. Code will be available at https://github.com/SitongGong/GROVE.

视频记忆可穿戴系统分层记忆主动助手

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。