让手机端实时看长视频不再卡顿，内存占用直降94%
InfiniPot-V: Memory-Constrained KV Cache Compression for Streaming Video Understanding
- 流式视频编码时动态压缩键值缓存，不依赖完整视频预加载
- 峰值显存降低94%，支持多轮对话仍保持准确率
- 无需训练、不需用户查询信息，适合移动端和边缘设备
现代多模态大模型可处理长达一小时的视频，但其键值（KV）缓存随时间线性增长，迅速超出手机、AR眼镜和边缘机器人的固定内存。以往压缩方法要么假设视频和用户查询可离线获取，要么必须先构建完整缓存，导致内存仍随流长增长。InfiniPot-V 是首个无需训练、与查询无关的框架，强制实现长度无关的内存上限。在视频编码过程中监测缓存，一旦达到用户设定阈值，即执行轻量压缩：(i) 通过时序冗余（TaR）度量去除时间冗余帧；(ii) 通过值范数（VaN）排序保留语义关键帧。在四个开源 MLLM 和四个长视频/流式视频基准上，InfiniPot-V 将峰值 GPU 显存降低高达 94%，维持实时生成，且准确率不低于全缓存版本——即使在多轮对话中亦然。该方案无须重训练或查询信息，彻底解决 KV 缓存瓶颈，推动终端设备实现流式视频助手。
原文摘要 · Abstract (English)
Modern multimodal large language models (MLLMs) can reason over hour-long video, yet their key-value (KV) cache grows linearly with time-quickly exceeding the fixed memory of phones, AR glasses, and edge robots. Prior compression schemes either assume the whole video and user query are available offline or must first build the full cache, so memory still scales with stream length. InfiniPot-V is the first training-free, query-agnostic framework that enforces a hard, length-independent memory cap for streaming video understanding. During video encoding it monitors the cache and, once a user-set threshold is reached, runs a lightweight compression pass that (i) removes temporally redundant tokens via Temporal-axis Redundancy (TaR) metric and (ii) keeps semantically significant tokens via Value-Norm (VaN) ranking. Across four open-source MLLMs and four long-video and streaming-video benchmarks, InfiniPot-V cuts peak GPU memory by up to 94%, sustains real-time generation, and matches or surpasses full-cache accuracy-even in multi-turn dialogues. By dissolving the KV cache bottleneck without retraining or query knowledge, InfiniPot-V closes the gap for on-device streaming video assistants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。