首个去中心化训练的扩散模型，无需集中算力即可生成高质量图像。
Paris: A Decentralized Trained Open-Weight Diffusion Model
- 8个独立专家模型分训，无梯度同步，通过动态路由选型。
- 用14倍少数据、16倍少算力达成与集中训练相当的生成质量。
- 适合想在异构设备上部署扩散模型的研究者与开发者。
我们提出Paris，首个完全通过去中心化计算预训练的公开扩散模型。Paris证明了高质量文本到图像生成无需集中式基础设施。该模型由8个独立训练的专家扩散模型（每模型参数量129M-605M）组成，训练中无梯度、参数或中间激活同步。我们采用分布式扩散训练框架，将数据划分为语义连贯的聚类，每个专家独立优化其子集，整体逼近完整分布。推理时，轻量级Transformer路由动态选择合适专家，生成质量接近集中式基线。去同步设计使模型可在异构硬件上训练，无需专用互联。实证表明，Paris在移除大型GPU集群需求的同时保持生成质量，相较之前去中心化基线，仅需14×更少训练数据和16×更少算力。
原文摘要 · Abstract (English)
We present Paris, the first publicly released diffusion model pre-trained entirely through decentralized computation. Paris demonstrates that high-quality text-to-image generation can be achieved without centrally coordinated infrastructure. Paris is open for research and commercial use. Paris required implementing our Distributed Diffusion Training framework from scratch. The model consists of 8 expert diffusion models (129M-605M parameters each) trained in complete isolation with no gradient, parameter, or intermediate activation synchronization. Rather than requiring synchronized gradient updates across thousands of GPUs, we partition data into semantically coherent clusters where each expert independently optimizes its subset while collectively approximating the full distribution. A lightweight transformer router dynamically selects appropriate experts at inference, achieving generation quality comparable to centrally coordinated baselines. Eliminating synchronization enables training on heterogeneous hardware without specialized interconnects. Empirical validation confirms that Paris's decentralized training maintains generation quality while removing the dedicated GPU cluster requirement for large-scale diffusion models. Paris achieves this using 14$\times$ less training data and 16$\times$ less compute than the prior decentralized baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。