arXiv:2410.10629cs.CV2024-10被引 347

Sana实现4096×4096高清图像快速生成,可部署在笔记本显卡上。

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers

  • 采用32倍压缩自编码器与线性注意力机制,降低计算开销。
  • 0.6B参数模型生成1024×1024图像不足1秒,速度超大模型100倍。
  • 适合低资源场景的高效图像生成,尤其适合个人创作者。

我们提出Sana,一个文本到图像生成框架,可高效生成最高4096×4096分辨率的图像。Sana在保持强文本-图像对齐的前提下,实现高速生成,且可在笔记本GPU上部署。核心设计包括:(1)深度压缩自编码器:训练出压缩比达32×的自编码器,显著减少潜在特征数量;(2)线性DiT:将扩散模型中的所有注意力替换为线性注意力,在高分辨率下更高效且不损失质量;(3)仅解码器文本编码器:用小型解码器语言模型替代T5,结合上下文学习增强图文对齐;(4)高效训练与采样:提出Flow-DPM-Solver以减少采样步数,并通过高效标题标注与选择加速收敛。结果表明,Sana-0.6B模型在性能上媲美大型扩散模型(如Flux-12B),体积小20倍,吞吐量快100倍以上。Sana-0.6B可在16GB笔记本显卡上运行,生成1024×1024图像耗时不足1秒。该系统实现低成本内容创作。代码与模型将公开发布。

原文摘要 · Abstract (English)

We introduce Sana, a text-to-image framework that can efficiently generate images up to 4096$\times$4096 resolution. Sana can synthesize high-resolution, high-quality images with strong text-image alignment at a remarkably fast speed, deployable on laptop GPU. Core designs include: (1) Deep compression autoencoder: unlike traditional AEs, which compress images only 8$\times$, we trained an AE that can compress images 32$\times$, effectively reducing the number of latent tokens. (2) Linear DiT: we replace all vanilla attention in DiT with linear attention, which is more efficient at high resolutions without sacrificing quality. (3) Decoder-only text encoder: we replaced T5 with modern decoder-only small LLM as the text encoder and designed complex human instruction with in-context learning to enhance the image-text alignment. (4) Efficient training and sampling: we propose Flow-DPM-Solver to reduce sampling steps, with efficient caption labeling and selection to accelerate convergence. As a result, Sana-0.6B is very competitive with modern giant diffusion model (e.g. Flux-12B), being 20 times smaller and 100+ times faster in measured throughput. Moreover, Sana-0.6B can be deployed on a 16GB laptop GPU, taking less than 1 second to generate a 1024$\times$1024 resolution image. Sana enables content creation at low cost. Code and model will be publicly released.

图像生成扩散模型轻量化高效推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。