用扩散模型生成语义分割小数据集,压缩99%仍保持高精度。
D3S2: Diffusion-Guided Dataset Distillation for Semantic Segmentation

- 分两阶段:先选代表性标签图,再用扩散模型生成对齐图像
- 1%压缩率下ADE20K上达24.99% mIoU,比随机选好9.34%以上
- 适合需要高效训练的小数据场景,如边缘设备部署
数据蒸馏旨在将大规模数据集压缩为紧凑的合成数据集,同时保持训练效果。然而,现有研究主要聚焦图像分类,对语义分割等密集预测任务关注不足。本文识别出分割数据蒸馏的三大挑战:(i) 类别长尾分布,(ii) 图像与密集标签间的像素级严格对齐需求,(iii) 高分辨率数据与复杂模型优化带来的高计算成本。为此,我们提出D3S2——一种面向语义分割的扩散引导数据蒸馏框架。方法采用两阶段设计:在类别平衡掩码选择阶段,通过贪心策略构建代表性的掩码集,优先覆盖低频类别;在扩散引导图像生成阶段,利用预训练的布局到图像扩散模型生成条件图像,自然保证空间对齐。为进一步提升合成数据的训练价值,引入有指导的扩散采样,包含两个互补目标:分割一致性损失以保证像素级对齐,类级特征匹配损失以对齐各层的类别特征统计。大量实验表明D3S2具有显著优势。尤其在1%的极低压缩率下,其在ADE20K和COCO-Stuff上使用Mask2Former (Swin-S) 分别取得24.99%和35.49%的mIoU,优于随机选择9.34%和5.70%。
原文摘要 · Abstract (English)
Dataset distillation (DD) aims to compress large-scale datasets into compact synthetic sets while preserving training efficacy. However, existing studies mainly focus on image classification, leaving dense prediction tasks such as semantic segmentation largely underexplored. In this work, we identify three key challenges for segmentation DD: (i) long-tailed class imbalance, (ii) the need for strict pixel-wise alignment between images and dense labels, and (iii) the high computational cost of optimizing high-resolution data with complex models. To address these challenges, we propose D3S2, a Diffusion-guided Dataset Distillation framework for Semantic Segmentation. Our method adopts a two-stage design. In Class-Balanced Mask Selection, we construct a representative mask set via a greedy strategy that prioritizes underrepresented classes. In Diffusion-Guided Image Synthesis, we employ a pretrained layout-to-image diffusion model to generate images conditioned on the selected masks, naturally ensuring spatial alignment. To further enhance the training utility of synthesized data, we introduce guided diffusion sampling with two complementary objectives: a segmentation-consistency loss for pixel-level alignment, and a class-wise feature matching loss for aligning per-class feature statistics across layers. Extensive experiments demonstrate the superiority of D3S2. Notably, at an extremely compression rate of 1%, our method achieves 24.99% and 35.49% mIoU on ADE20K and COCO-Stuff with Mask2Former (Swin-S), outperforming random selection by 9.34% and 5.70%, respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。