arXiv:2502.05027cs.CV2025-02被引 1

针对标签错误数据,提出可信度感知的蒸馏方法提升小样本数据压缩效果。

Trust-Aware Diversion for Data-Effective Distillation

  • 通过内外双循环机制区分可信与不可信样本,引导蒸馏聚焦可信数据。
  • 在三种标签错误场景下,显著提升CIFAR10/100和Tiny ImageNet的蒸馏性能。
  • 适合标签质量不高的真实场景,提升数据蒸馏的鲁棒性与实用性。

数据蒸馏将大规模数据集压缩为少量合成子集以保留关键信息。现有方法假设所有样本标签正确,限制了其在标签错误普遍存在的现实场景中的应用。错误标签引入不可信信息,误导模型优化过程。为此,我们提出可信度感知分流(Trust-Aware Diversion, TAD)的数据蒸馏方法。TAD采用迭代双循环优化框架实现高效蒸馏:外层循环将数据划分为可信与不可信空间,引导蒸馏优先使用可信样本,确保蒸馏过程的可靠性;内层循环通过重校准不可信样本,将其转化为可利用的优质样本。双循环迭代优化,逐步扩大可信空间、缩小不可信空间。实验表明,该方法在三个主流基准(CIFAR10、CIFAR100、Tiny ImageNet)的三种挑战性标签错误设置(对称、非对称、真实世界)下,显著提升了现有数据蒸馏方法的性能。

原文摘要 · Abstract (English)

Dataset distillation compresses a large dataset into a small synthetic subset that retains essential information. Existing methods assume that all samples are perfectly labeled, limiting their real-world applications where incorrect labels are ubiquitous. These mislabeled samples introduce untrustworthy information into the dataset, which misleads model optimization in dataset distillation. To tackle this issue, we propose a Trust-Aware Diversion (TAD) dataset distillation method. Our proposed TAD introduces an iterative dual-loop optimization framework for data-effective distillation. Specifically, the outer loop divides data into trusted and untrusted spaces, redirecting distillation toward trusted samples to guarantee trust in the distillation process. This step minimizes the impact of mislabeled samples on dataset distillation. The inner loop maximizes the distillation objective by recalibrating untrusted samples, thus transforming them into valuable ones for distillation. This dual-loop iteratively refines and compensates for each other, gradually expanding the trusted space and shrinking the untrusted space. Experiments demonstrate that our method can significantly improve the performance of existing dataset distillation methods on three widely used benchmarks (CIFAR10, CIFAR100, and Tiny ImageNet) in three challenging mislabeled settings (symmetric, asymmetric, and real-world).

数据蒸馏标签噪声可信度建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。