arXiv:2606.20076cs.CVcs.AI2026-06

提出可学习全局合并的变长分词器,提升扩散模型质量与计算效率的平衡。

Variable-Length Tokenization via Learnable Global Merging for Diffusion Transformers

论文配图:Variable-Length Tokenization via Learnable Global Merging for Diffusion Transformers
图 1 · 摘自论文原文
  • 通过可学习的全局合并机制调节分词长度,避免位置依赖问题。
  • 在ImageNet 256×256上实现更优的gFID-计算权衡表现。
  • 适合追求高效高质图像生成的开发者与研究者使用。

潜在扩散模型(LDMs)已成为视觉合成的主流,但其质量与计算效率的权衡受限于分词器固定的压缩比。变长分词器(VLTs)通过动态调整分词数量实现自适应压缩,使扩散模型能灵活平衡质量与计算开销。然而,传统VLTs通过截断有序分词序列来调节长度,导致分词语义依赖位置,破坏跨长度表征对齐,引发潜在分布偏移,阻碍单一变长扩散模型有效运行。为此,本文提出一种基于分词合并的新型变长分词器。我们证明,鼓励相似分词合并可使扩散转换器按合并模式直接实现跨长度表征对齐。由于传统合并方法依赖数据,生成时无法获取合并模式,因此引入可学习全局合并(learnable global merging),其数据无关特性确保与扩散转换器兼容。在ImageNet 256×256图像生成任务中,集成该分词器的扩散模型相比先前VLT方法实现了更优的gFID-计算权衡。代码已开源。

原文摘要 · Abstract (English)

Latent Diffusion Models (LDMs) have become dominant in visual synthesis, but their quality-compute trade-off is largely constrained by the tokenizer's fixed compression ratio. Variable-length tokenizers (VLTs) promise adaptive compression by varying token counts, allowing diffusion models to flexibly balance quality and compute. However, conventional VLTs modulate length by truncating ordered token sequences, which makes token semantics depend on token position and breaks representational alignment across lengths. This leads to a cross-length shift in the latent distribution that hinders a single variable-length diffusion model from operating effectively. To address this, we propose a novel variable-length tokenizer that modulates length by merging tokens. We show that encouraging similar tokens to merge enables direct cross-length representation alignment when the diffusion transformer operates according to the merging pattern. Since conventional merging methods are data-dependent, making the merging pattern inaccessible during generation, we introduce learnable global merging, which is data-independent, to ensure compatibility with diffusion transformers. On ImageNet 256$\times$256 generation, our merging-based variable-length tokenizer integrated with a diffusion transformer achieves a superior gFID-compute trade-off compared to prior VLT methods. Code is available at [this https URL](https://github.com/movinghoon/lgm)

扩散模型变长分词生成模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。