用Mamba架构和通道拆分量化，提升视频离散化效率与质量。
MambaVideo for Discrete Video Tokenization with Channel-Split Quantization
- 基于Mamba的编码器-解码器结构，突破传统序列模型局限。
- 通道拆分量化使量化特征表达力更强，且保持相同标记数量。
- 在多数据集上超越3D卷积与Transformer方法，适合自回归视频生成。
由于视频数据维度高，离散视频标记化对高效自回归生成建模至关重要。本文提出一种新的先进离散视频标记器，包含两项关键贡献：首先，设计了一种基于Mamba的编码器-解码器架构，克服了先前序列型标记器的局限性；其次，引入一种新量化方案——通道拆分量化，显著增强量化潜变量的表征能力，同时保持标记数量不变。该模型在多个数据集上达到新基准，性能优于因果3D卷积和Transformer基线方法。实验进一步验证其作为自回归视频生成标记器的鲁棒性。
原文摘要 · Abstract (English)
Discrete video tokenization is essential for efficient autoregressive generative modeling due to the high dimensionality of video data. This work introduces a state-of-the-art discrete video tokenizer with two key contributions. First, we propose a novel Mamba-based encoder-decoder architecture that overcomes the limitations of previous sequencebased tokenizers. Second, we introduce a new quantization scheme, channel-split quantization, which significantly enhances the representational power of quantized latents while preserving the token count. Our model sets a new state-of-the-art, outperforming both causal 3D convolutionbased and Transformer-based approaches across multiple datasets. Experimental results further demonstrate its robustness as a tokenizer for autoregressive video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。