用半监督蒸馏让小模型高效生成可控图像，节省资源又保质量。
REFINE-CONTROL: A Semi-supervised Distillation Method For Conditional Image Generation
- 通过三级知识融合损失，分层迁移大模型知识到小模型。
- 仅用少量标注数据+大量无标注数据，实现性能接近全监督方法。
- 适合边缘设备部署，降低计算开销和隐私风险。
条件图像生成模型通过文本控制实现了高度定制化的图像生成，但其高资源消耗及标注数据稀缺问题限制了在边缘设备上的部署，带来巨大成本与隐私隐患。为此，我们提出 Refine-Control，一种半监督蒸馏框架。具体而言，引入三层次知识融合损失，以分层转移不同粒度的知识；为提升泛化能力并缓解数据稀缺，采用结合有标签与无标签数据的半监督蒸馏方法。实验表明，Refine-Control 在显著降低计算成本与延迟的同时，仍保持高保真生成能力和可控性，各项量化指标表现优异。
原文摘要 · Abstract (English)
Conditional image generation models have achieved remarkable results by leveraging text-based control to generate customized images. However, the high resource demands of these models and the scarcity of well-annotated data have hindered their deployment on edge devices, leading to enormous costs and privacy concerns, especially when user data is sent to a third party. To overcome these challenges, we propose Refine-Control, a semi-supervised distillation framework. Specifically, we improve the performance of the student model by introducing a tri-level knowledge fusion loss to transfer different levels of knowledge. To enhance generalization and alleviate dataset scarcity, we introduce a semi-supervised distillation method utilizing both labeled and unlabeled data. Our experiments reveal that Refine-Control achieves significant reductions in computational cost and latency, while maintaining high-fidelity generation capabilities and controllability, as quantified by comparative metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。