arXiv:2606.17590cs.CV2026-06被引 1

TivTok通过分离不变与变化信息,大幅降低视频生成的令牌数量和计算成本。

TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenization

论文配图:TivTok: Broadcasting Time-Invariant Tokens for Scalable Video Tokenization
图 1 · 摘自论文原文
  • 将视频分为空间不变(TIV)和帧间变化(TV)令牌,实现信息复用
  • 在128帧视频上压缩效率提升2.91倍,仅需1.1%的令牌数
  • 适合长视频生成、低资源部署及高效扩散模型应用

视频标记化是实现可扩展视频生成的基础,令牌数量直接决定计算成本和可建模视频长度。现有标记器主要通过减少令牌数量来提升可扩展性,但往往重复表示静态背景、物体外观等持久内容。本文提出TivTok(时间不变标记器),一种感知复用的视频标记器,使持久信息可在时间上复用。TivTok以时间不变(TIV)令牌编码跨帧共享信息,时间变化(TV)令牌编码帧级残差。为实现该分解,引入范围诱导分解(SIF),赋予两类令牌不同注意力范围:TIV令牌关注整个片段,而每个TV令牌仅访问对应帧及其关联的TIV令牌。解码时,不变广播(IB)在帧与块间复用相同TIV令牌,实现并行重建与长视频标记化。实验表明,TivTok在标准16×256×256基准上取得rFVD 12.65,相比基线压缩效率提升2.91倍,且仅需下采样型标记器1.1%的令牌数。

原文摘要 · Abstract (English)

Video tokenization is fundamental to scalable video generation, as the number of tokens directly determines the computational cost and the length of videos that can be modeled. Existing tokenizers mainly improve scalability by compressing videos into fewer tokens, but they often continue to represent persistent content, such as static backgrounds and consistent object appearances, repeatedly across frames and chunks. In this paper, we propose \textbf{TivTok} (\textit{Time-Invariant Tokenizer}), a reuse-aware video tokenizer that makes persistent information reusable across time. TivTok represents a clip with Time-Invariant (TIV) tokens that encode information shared across frames and Time-Variant (TV) tokens that encode frame-specific residuals. To obtain this factorization, we introduce Scope-Induced Factorization (SIF), which assigns different attention scopes to the two token groups: TIV tokens attend to the full clip, whereas each TV token only accesses its corresponding frame together with the TIV tokens. In the decoder, Invariant Broadcasting (IB) reuses the same TIV tokens across frames and chunks for parallel reconstruction and long-video tokenization. Experiments show that TivTok achieves an rFVD of 12.65 on the standard $16{\times}256{\times}256$ benchmark and improves compression efficiency by 2.91$\times$ for 128-frame videos compared with the evaluated baselines, while using only 1.1\% of the tokens required by downsample-based tokenizers in our evaluation.

视频生成令牌化扩散模型压缩效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。