用32个连续标记压缩视觉特征，让扩散模型更高效
Laminating Representation Autoencoders for Efficient Diffusion
- 将DINOv2的密集特征压缩为32个连续令牌
- 图像生成效率提升8倍，训练耗能减少4.5倍
- 适合追求高效扩散模型的开发者和研究者
近期研究表明，扩散模型可直接在自监督学习（SSL）patch特征上生成高质量图像，而非像素空间隐变量。然而，DINOv2等编码器产生的密集patch网格存在显著冗余，导致扩散过程成本过高。本文提出FlatDINO，一种变分自编码器，将该表示压缩为仅32个连续令牌的一维序列——序列长度减少8倍，总维度压缩48倍。在ImageNet 256x256上，基于FlatDINO隐变量训练的DiT-XL模型，在使用分类器无关引导时达到gFID 1.80，且前向传播减少8倍浮点运算量，训练步骤最多节省4.5倍浮点运算量。这些是初步结果，工作仍在进行中。
原文摘要 · Abstract (English)
Recent work has shown that diffusion models can generate high-quality images by operating directly on SSL patch features rather than pixel-space latents. However, the dense patch grids from encoders like DINOv2 contain significant redundancy, making diffusion needlessly expensive. We introduce FlatDINO, a variational autoencoder that compresses this representation into a one-dimensional sequence of just 32 continuous tokens -an 8x reduction in sequence length and 48x compression in total dimensionality. On ImageNet 256x256, a DiT-XL trained on FlatDINO latents achieves a gFID of 1.80 with classifier-free guidance while requiring 8x fewer FLOPs per forward pass and up to 4.5x fewer FLOPs per training step compared to diffusion on uncompressed DINOv2 features. These are preliminary results and this work is in progress.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。