arXiv:2501.06405cs.CVcs.AI2025-01被引 4

用关键图像块生成压缩数据集,提升模型泛化能力。

FocusDD: Real-World Scene Infusion for Robust Dataset Distillation

  • 通过ViT提取关键图像块,合成带多目标的压缩图像
  • 100张/类时,ResNet50准确率达71.0%,超越现有方法2.8%
  • 首次将压缩数据集用于目标检测,支持多种网络架构

数据集蒸馏被提出用于高效压缩真实世界数据集。然而,其在大规模、高分辨率数据集上表现不佳,限制了实用性。本文提出一种与分辨率无关的数据集蒸馏方法——焦点数据蒸馏(FocusDD),通过识别关键信息块来实现蒸馏数据的多样性和真实性,确保不同网络架构下的泛化能力。具体地,FocusDD利用预训练视觉变压器(ViT)提取关键图像块,并将其合成为单个蒸馏图像。这些包含多个目标的蒸馏图像不仅适用于分类任务,也适用于目标检测等密集任务。为进一步提升泛化性,每个合成图像还附加原始图像的下采样视图。在ImageNet-1K数据集上,每类100张图像(IPC)时,ResNet50和MobileNet-v2的验证准确率分别为71.0%和62.6%,分别优于当前最优方法2.8%和4.7%。值得注意的是,FocusDD是首个将蒸馏数据集用于目标检测的方法。在COCO2017数据集上,以IPC=50为例,YOLOv11n和YOLOv11s的mAP分别为24.4%和32.1%,进一步验证了该方法的有效性。

原文摘要 · Abstract (English)

Dataset distillation has emerged as a strategy to compress real-world datasets for efficient training. However, it struggles with large-scale and high-resolution datasets, limiting its practicality. This paper introduces a novel resolution-independent dataset distillation method Focus ed Dataset Distillation (FocusDD), which achieves diversity and realism in distilled data by identifying key information patches, thereby ensuring the generalization capability of the distilled dataset across different network architectures. Specifically, FocusDD leverages a pre-trained Vision Transformer (ViT) to extract key image patches, which are then synthesized into a single distilled image. These distilled images, which capture multiple targets, are suitable not only for classification tasks but also for dense tasks such as object detection. To further improve the generalization of the distilled dataset, each synthesized image is augmented with a downsampled view of the original image. Experimental results on the ImageNet-1K dataset demonstrate that, with 100 images per class (IPC), ResNet50 and MobileNet-v2 achieve validation accuracies of 71.0% and 62.6%, respectively, outperforming state-of-the-art methods by 2.8% and 4.7%. Notably, FocusDD is the first method to use distilled datasets for object detection tasks. On the COCO2017 dataset, with an IPC of 50, YOLOv11n and YOLOv11s achieve 24.4% and 32.1% mAP, respectively, further validating the effectiveness of our approach.

数据集蒸馏视觉任务目标检测ViT

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。