根据任务难度生成更优的合成数据集,提升分类性能。
Task-Specific Generative Dataset Distillation with Difficulty-Guided Sampling
- 基于任务难度设计采样策略,优化合成数据分布。
- 在多个分类任务上实现与原始数据集相当的准确率。
- 适合需要高效训练的下游任务场景,如资源受限环境。
为减少深度神经网络对大规模数据集的依赖,数据蒸馏旨在生成紧凑且高质量的合成数据集,使其性能可媲美原始数据集。生成模型的引入显著推动了该领域的发展。然而,现有方法主要关注合成数据与原始数据的对齐,常忽略对下游任务至关重要的任务特定信息。本文针对分类任务,提出一种融合难度概念的任务特定采样策略,通过匹配原始数据集的难度分布来生成最终数据集。采样前对难度分布进行对数变换以纠正偏差。大量实验表明,该方法有效,且在其他下游任务中也具应用潜力。代码已公开于 https://github.com/SumomoTaku/DiffGuideSamp。
原文摘要 · Abstract (English)
To alleviate the reliance of deep neural networks on large-scale datasets, dataset distillation aims to generate compact, high-quality synthetic datasets that can achieve comparable performance to the original dataset. The integration of generative models has significantly advanced this field. However, existing approaches primarily focus on aligning the distilled dataset with the original one, often overlooking task-specific information that can be critical for optimal downstream performance. In this paper, focusing on the downstream task of classification, we propose a task-specific sampling strategy for generative dataset distillation that incorporates the concept of difficulty to consider the requirements of the target task better. The final dataset is sampled from a larger image pool with a sampling distribution obtained by matching the difficulty distribution of the original dataset. A logarithmic transformation is applied as a pre-processing step to correct for distributional bias. The results of extensive experiments demonstrate the effectiveness of our method and suggest its potential for enhancing performance on other downstream tasks. The code is available at https://github.com/SumomoTaku/DiffGuideSamp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。