arXiv:2603.25144cs.CVcs.AI2026-03中稿 · ECCV被引 2

针对细粒度数据集压缩,提出新框架提升样本区分度。

FD$^2$: A Dedicated Framework for Fine-Grained Dataset Distillation

  • 通过定位判别区域构建细粒度表征,优化类内差异与类间区分。
  • 在多个细粒度数据集上性能超越现有方法,提升显著且具通用性。
  • 适合需要高精度分类的细粒度图像任务,如物种识别、医学影像。

数据集蒸馏(DD)将大规模训练集压缩为小规模合成集,降低存储与训练成本,并在通用基准上表现优异。解耦型DD进一步通过预训练、样本蒸馏和软标签生成三阶段提升效率。然而,现有解耦方法多依赖粗粒度类别监督,类内样本优化方式相似,在细粒度数据集上常导致蒸馏样本保留大类内差异、类间差异微弱,且同类别样本过于相似,削弱局部判别线索,影响识别效果。为此,本文提出专用于细粒度数据集蒸馏的FD²框架。该框架通过定位判别区域并构建细粒度表示实现蒸馏。预训练阶段采用反事实注意力学习聚合判别特征以更新类别原型;蒸馏阶段引入细粒度特征约束,使每个样本贴近所属类别原型并远离其他类别,同时通过相似性约束促进同类别样本间的注意力多样性。在多个细粒度及通用数据集上的实验表明,FD²可无缝集成至解耦型DD,多数场景下性能均有提升,展现良好迁移能力。代码已开源:https://github.com/Guang000/FD2。

原文摘要 · Abstract (English)

Dataset distillation (DD) compresses a large training set into a small synthetic set, reducing storage and training cost, and has shown strong results on general benchmarks. Decoupled DD further improves efficiency by splitting the pipeline into pretraining, sample distillation, and soft-label generation. However, existing decoupled methods largely rely on coarse class-label supervision and optimize samples within each class in a nearly identical manner. On fine-grained datasets, this often yields distilled samples that (i) retain large intra-class variation with subtle inter-class differences and (ii) become overly similar within the same class, limiting localized discriminative cues and hurting recognition. To solve the above-mentioned problems, we propose FD$^{2}$, a dedicated framework for Fine-grained Dataset Distillation. FD$^{2}$ localizes discriminative regions and constructs fine-grained representations for distillation. During pretraining, counterfactual attention learning aggregates discriminative representations to update class prototypes. During distillation, a fine-grained characteristic constraint aligns each sample with its class prototype while repelling others, and a similarity constraint diversifies attention across same-class samples. Experiments on multiple fine-grained and general datasets show that FD$^{2}$ integrates seamlessly with decoupled DD and improves performance in most settings, indicating strong transferability. Code is available at https://github.com/Guang000/FD2.

数据集蒸馏细粒度识别图像分类模型压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。