arXiv:2603.13346cs.CVcs.AI2026-03AAAI被引 2

用2比特量化压缩数据凝练结果,准确率翻倍且无需重训练。

Post Training Quantization for Efficient Dataset Condensation

  • 按图像块分组量化,减少信息损失
  • 2比特下准确率从26%提升至54%
  • 可直接用于各类数据凝练方法

数据凝练(DC)将大模型知识压缩到小数据集,加速训练并降低存储需求。然而,现有方法大多忽略量化在进一步压缩存储上的潜力。本文首次探索数据凝练中的后训练量化,证明其在不增加训练成本的前提下有效降低存储开销。但发现极低比特(如2比特)时,传统量化导致表征质量严重下降。为此,提出基于图像块的后训练量化方法,确保局部量化损失最小;为降低量化参数开销,采用量化感知聚类合并相似块;并引入修正模块对齐原始图像与反量化图像分布,补偿量化误差。该方法为即插即用框架,适用于多种DC生成的合成图像。在CIFAR-10/100、Tiny ImageNet及ImageNet子集等多基准测试中,本方法在相同存储约束下持续优于现有工作。尤其在极端压缩场景(如IPC=1),测试准确率几乎翻倍(26.0% → 54.1%),且直接作用于2比特图像,无需额外蒸馏。

原文摘要 · Abstract (English)

Dataset Condensation (DC) distills knowledge from large datasets into smaller ones, accelerating training and reducing storage requirements. However, despite notable progress, prior methods have largely overlooked the potential of quantization for further reducing storage costs. In this paper, we take the first step to explore post-training quantization in dataset condensation, demonstrating its effectiveness in reducing storage size while maintaining representation quality without requiring expensive training cost. However, we find that at extremely low bit-widths (e.g., 2-bit), conventional quantization leads to substantial degradation in representation quality, negatively impacting the networks trained on these data. To address this, we propose a novel \emph{patch-based post-training quantization} approach that ensures localized quantization with minimal loss of information. To reduce the overhead of quantization parameters, especially for small patch sizes, we employ quantization-aware clustering to identify similar patches and subsequently aggregate them for efficient quantization. Furthermore, we introduce a refinement module to align the distribution between original images and their dequantized counterparts, compensating for quantization errors. Our method is a plug-and-play framework that can be applied to synthetic images generated by various DC methods. Extensive experiments across diverse benchmarks including CIFAR-10/100, Tiny ImageNet, and ImageNet subsets demonstrate that our method consistently outperforms prior works under the same storage constraints. Notably, our method nearly \textbf{doubles the test accuracy} of existing methods at extreme compression regimes (e.g., 26.0\% $\rightarrow$ 54.1\% for DM at IPC=1), while operating directly on 2-bit images without additional distillation.

数据凝练量化高效存储2比特

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。