让高容量扩散模型教低配模型,突破空间不匹配瓶颈
Cross-Space Distillation: Teaching One-Step Students with Modern Diffusion Teachers

- 用轻量级桥接模块实现跨空间特征对齐
- 使SD 1.5在单步生成中HPSv3得分从5.4提升至9.4
- 无需修改学生模型,适配部署场景
现代单步扩散模型通过分布式时间步蒸馏获得优异质量,但依赖教师与学生处于同一潜在空间的假设。该共享空间约束阻碍了高容量教师(如SD 3.5、Flux)向紧凑型部署模型(如SD 1.5)的知识迁移,因后者潜在分辨率和VAE参数化不同。本文提出跨空间蒸馏(Cross-Space Distillation),允许教师与学生在潜在分辨率和VAE空间上均存在差异。为此引入轻量级桥接模块(Bridge),将学生潜在表示映射至教师空间,不修改学生主干。Bridge结合冻结的学生VAE解码器作为空间先验与可学习投影器,通过潜在重建与注意力保真目标训练以实现稳定对齐。在多种现代教师模型下,该方法显著提升紧凑型单步学生模型性能:例如将SD 1.5的HPSv3得分从5.4提升至9.4,同时保持单步推理、低延迟与广泛生态兼容性。结果表明,异构大模型可通过轻量级潜在空间接口高效蒸馏为可部署骨干。
原文摘要 · Abstract (English)
Modern one-step diffusion models achieve impressive quality through distribution-based timestep distillation. Yet, they rely on a critical assumption: Teacher and Student must inhabit the same latent space. This Shared-Space constraint prevents knowledge transfer from modern high-capacity Teachers (e.g., SD 3.5 and Flux) into compact, deployment-friendly Students such as SD 1.5, whose latent resolution and VAE parameterization differ from the Teacher. We formalize this overlooked regime as Cross-Space Distillation, where Teacher and Student differ in both latent resolution and VAE space. To enable distillation under this mismatch, we introduce the Bridge, a lightweight latent interface that maps Student latents into the Teacher space without modifying the Student backbone. Bridge combines a frozen Student VAE decoder as a spatial prior with a compact learnable projector, and is trained with latent reconstruction and attention fidelity objectives for stable Teacher-space alignment. Across diverse modern Teachers, Bridge enables substantial gains for compact one-step Students; for example, it improves SD 1.5 from 5.4 to 9.4 HPSv3 while preserving one-step inference, low latency, and broad ecosystem compatibility. These results show that heterogeneous large Teachers can be distilled into efficient, deployable backbones through a lightweight latent-space interface.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。