arXiv:2608.04866cs.CV2026-08

用对象叙事结构减少视频大模型的视觉令牌占用。

Persistent Object Narratives for Token-Efficient Video Language Models

论文配图:Persistent Object Narratives for Token-Efficient Video Language Models
图 1 · 摘自论文原文
  • 将视频划分为持续对象槽,用紧凑令牌表示对象状态。
  • 仅需144个视觉令牌,与帧数无关,仍保持高精度。
  • 适合追求高效推理的视频理解系统开发者。

视频大语言模型(Video-LLMs)在开放域视频理解上取得显著进展,但其视觉接口仍存在令牌消耗大、缺乏显式时序结构的问题。本文提出SlotNarrative,一种基于槽的接口,将视频组织为由紧凑对象状态令牌表示的持续对象叙事。不同于先压缩帧级特征再建立时序对应的方法,SlotNarrative首先将视觉特征分组为类对象槽,并通过轻量、无参数的记忆模块,结合多种互补匹配线索,将重复观测关联到片段级对象条目。每个保留条目序列化为两类令牌:身份令牌(总结对象外观)和一组状态令牌(编码段级外观、几何、可见性及轨迹信息)。该设计使冻结的Video-LLM仅需144个分配的视觉令牌位置,且不随采样帧数变化。在多个数据集上,相比先前紧凑接口,SlotNarrative在准确率与视觉令牌数量之间实现了更优权衡。实验验证了持续对象叙事作为紧凑、结构化、时序有序的Video-LLM视觉接口的有效性。代码将公开。

原文摘要 · Abstract (English)

Video large language models (Video-LLMs) have made strong progress in open-ended video understanding. However, their visual interfaces remain token-intensive and provide limited explicit structure for linking recurring object evidence across time. We introduce SlotNarrative, a slot-based interface that organizes a video into persistent object narratives represented by compact object-state tokens. Rather than compressing frame-wise features before establishing temporal correspondence, SlotNarrative first groups visual features into object-like slots and then associates recurring observations with clip-level object entries through a lightweight, parameter-free memory that integrates multiple complementary matching cues. Each retained entry is serialized into two token types: an identity token that summarizes persistent object appearance and a set of state tokens that encode segment-level appearance, geometry, visibility, and trajectory information. This design yields an interface of only 144 allocated visual-token positions for a frozen Video-LLM, independent of the number of sampled frames. Across multiple datasets, SlotNarrative achieves a favorable trade-off between accuracy and visual-token count compared with prior compact Video-LLM interfaces. Experimental results establish persistent object narratives as a compact, structured, and temporally organized visual interface for Video-LLMs. Our code will be made publicly available.

视频理解令牌效率对象追踪

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。