arXiv:2603.06351cs.CVcs.AI2026-03被引 2

让图像生成模型自动分配计算资源,更省算力还更清晰。

DC-DiT: Adaptive Compute and Elastic Inference for Visual Generation via Dynamic Chunking

  • 用动态分块机制替代固定分块,按需压缩图像信息。
  • 推理时减少36.8%算力,FID提升37.8%,画质更优。
  • 支持灵活计算预算调整,适合部署在资源受限场景。

扩散Transformer依赖静态的图像分块方式,对平滑背景、细节区域、噪声早期步和后期优化阶段均使用相同令牌预算。本文提出动态分块扩散Transformer(DC-DiT),以端到端学习的编码器-路由器-解码器结构替代固定分块,通过可学习的分块机制自适应压缩二维输入为更短的令牌序列。该方法在可预测区域和噪声早期步骤中减少令牌数量,在细节区域和后期优化阶段增加令牌数量,实现无需监督的空间分割与时间自适应压缩策略。此外,路由器提供保留令牌的重要性排序,支持弹性推理:单个检查点可在不同计算预算下灵活评估,实现质量与计算量的平滑权衡。DC-DiT可由预训练DiT模型迁移而来,且兼容其他动态计算方法。在类别条件图像生成任务中,相比基线DiT模型,最多减少36.8%推理浮点运算量,同时将FID提升37.8%,在多种模型规模、分辨率与引导设置下均获得更强的质量-算力帕累托前沿。结果表明,自适应令牌化是提升视觉生成效率与推理灵活性的通用机制。

原文摘要 · Abstract (English)

Diffusion Transformers rely on static patchify tokenization, assigning the same token budget to smooth backgrounds, detailed object regions, noisy early timesteps, and late-stage refinements. We introduce the Dynamic Chunking Diffusion Transformer (DC-DiT), which replaces fixed patchification with a learned encoder-router-decoder scaffold that adaptively compresses the 2D input into a shorter token sequence through a chunking mechanism learned end-to-end with diffusion training. DC-DiT allocates fewer tokens to predictable regions and noisy timesteps, and more tokens to detailed regions and later refinement stages, yielding meaningful spatial segmentations and timestep-adaptive compression schedules without supervision. Furthermore, the router provides an importance ordering over retained tokens, enabling elastic inference: a single checkpoint can be evaluated at flexible compute budgets with a smooth quality-compute tradeoff. Additionally, DC-DiT can be upcycled from pretrained DiT checkpoints and is also compatible with orthogonal dynamic computation approaches. On class-conditional ImageNet generation, DC-DiT reduces inference FLOPs by up to 36.8% and improves FID by up to 37.8% over DiT baselines, yielding a stronger quality--compute Pareto frontier across model scales, resolutions, and guidance settings. More broadly, these results suggest that adaptive tokenization is a general mechanism for making visual generation both more efficient and more flexible at inference time.

图像生成扩散模型动态计算弹性推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。