arXiv:2507.02358cs.CVcs.AI2025-07ICCV被引 11

Hita让图像生成先看整体再看局部,提升质量与速度。

Hita: Holistic Tokenizer for Autoregressive Image Generation

  • 先用可学习的全局查询捕捉整体特征,再生成局部图像块。
  • 在ImageNet上达2.59 FID和281.9 IS,训练更快更准。
  • 适合追求高质量图像生成与零样本风格迁移的研究者。

传统自回归图像生成模型逐个生成视觉标记,难以捕捉标记序列间的全局关系。由于多数视觉分词器将局部图像块映射为潜在标记,全局信息受限。为此,我们提出新分词器Hita,采用整体到局部的分词策略,包含可学习的全局查询和局部块标记。Hita通过两项关键设计更好适配自回归生成过程:1)将全局标记置于序列开头,后接块级标记,并使用因果注意力保持对前序标记的关注;2)在解码前引入轻量融合模块,调控信息流并优先保留全局标记。大量实验表明,Hita加速了自回归生成器的训练速度,性能优于使用传统分词器的模型,在ImageNet上达到2.59 FID和281.9 IS。对全局表示的详细分析显示其能有效捕捉纹理、材质和形状等全局属性。此外,Hita在零样本风格迁移与图像修复任务中也表现优异。代码已公开于https://github.com/CVMI-Lab/Hita。

原文摘要 · Abstract (English)

Vanilla autoregressive image generation models generate visual tokens step-by-step, limiting their ability to capture holistic relationships among token sequences. Moreover, because most visual tokenizers map local image patches into latent tokens, global information is limited. To address this, we introduce \textit{Hita}, a novel image tokenizer for autoregressive (AR) image generation. It introduces a holistic-to-local tokenization scheme with learnable holistic queries and local patch tokens. Hita incorporates two key strategies to better align with the AR generation process: 1) {arranging} a sequential structure with holistic tokens at the beginning, followed by patch-level tokens, and using causal attention to maintain awareness of previous tokens; and 2) adopting a lightweight fusion module before feeding the de-quantized tokens into the decoder to control information flow and prioritize holistic tokens. Extensive experiments show that Hita accelerates the training speed of AR generators and outperforms those trained with vanilla tokenizers, achieving \textbf{2.59 FID} and \textbf{281.9 IS} on the ImageNet benchmark. Detailed analysis of the holistic representation highlights its ability to capture global image properties, such as textures, materials, and shapes. Additionally, Hita also demonstrates effectiveness in zero-shot style transfer and image in-painting. The code is available at \href{https://github.com/CVMI-Lab/Hita}{https://github.com/CVMI-Lab/Hita}.

图像生成自回归分词器全局建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。