arXiv:2505.22705cs.CVcs.MM2025-05被引 141

170亿参数模型秒出高清图,高效又开源。

HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformer

  • 用稀疏扩散Transformer+动态专家混合架构,降低计算开销。
  • 三版本适配不同需求,最快生成仅需数秒。
  • 支持指令编辑与交互式图像创作,适合研究与开发使用。

近期图像生成基础模型在提升质量的同时往往伴随计算复杂度和推理延迟的增加。为解决这一关键矛盾,我们提出HiDream-I1,一个拥有170亿参数的开源图像生成基础模型,在数秒内实现顶尖图像生成质量。该模型采用新型稀疏扩散Transformer(DiT)结构:首先通过双流解耦设计,由两个独立编码器分别处理图像和文本令牌;随后采用单流稀疏DiT结合动态专家混合(MoE)架构,以低成本方式触发多模态交互生成图像。为支持灵活访问,提供三种变体:HiDream-I1-Full、HiDream-I1-Dev 和 HiDream-I1-Fast。进一步地,我们拓展其能力,引入额外图像条件,实现精准指令式图像编辑,形成新的指令式图像编辑模型HiDream-E1。最终,整合文本到图像生成与指令式图像编辑功能,构建具备全交互能力的图像智能体HiDream-A1。为加速多模态AIGC研究,所有代码与模型权重已开源,可通过https://github.com/HiDream-ai/HiDream-I1 和 https://github.com/HiDream-ai/HiDream-E1 获取,功能可直接体验于 https://vivago.ai/studio。

原文摘要 · Abstract (English)

Recent advancements in image generative foundation models have prioritized quality improvements but often at the cost of increased computational complexity and inference latency. To address this critical trade-off, we introduce HiDream-I1, a new open-source image generative foundation model with 17B parameters that achieves state-of-the-art image generation quality within seconds. HiDream-I1 is constructed with a new sparse Diffusion Transformer (DiT) structure. Specifically, it starts with a dual-stream decoupled design of sparse DiT with dynamic Mixture-of-Experts (MoE) architecture, in which two separate encoders are first involved to independently process image and text tokens. Then, a single-stream sparse DiT structure with dynamic MoE architecture is adopted to trigger multi-model interaction for image generation in a cost-efficient manner. To support flexiable accessibility with varied model capabilities, we provide HiDream-I1 in three variants: HiDream-I1-Full, HiDream-I1-Dev, and HiDream-I1-Fast. Furthermore, we go beyond the typical text-to-image generation and remould HiDream-I1 with additional image conditions to perform precise, instruction-based editing on given images, yielding a new instruction-based image editing model namely HiDream-E1. Ultimately, by integrating text-to-image generation and instruction-based image editing, HiDream-I1 evolves to form a comprehensive image agent (HiDream-A1) capable of fully interactive image creation and refinement. To accelerate multi-modal AIGC research, we have open-sourced all the codes and model weights of HiDream-I1-Full, HiDream-I1-Dev, HiDream-I1-Fast, HiDream-E1 through our project websites: https://github.com/HiDream-ai/HiDream-I1 and https://github.com/HiDream-ai/HiDream-E1. All features can be directly experienced via https://vivago.ai/studio.

图像生成扩散模型高效架构指令编辑

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。