通过压缩图像颜色冗余，实现大规模数据集高效存储而不损训练效果
Dataset Color Quantization: A Training-Oriented Framework for Dataset-Level Compression
- 统一图像颜色调色板，减少颜色空间冗余
- 在极端压缩下仍保持模型训练性能，如ImageNet-1K上表现稳定
- 适合资源受限环境下的数据集部署与模型训练
大规模图像数据集是深度学习的基础，但其高存储需求给资源受限环境的部署带来挑战。现有方法通过丢弃样本缩小数据集，却常忽略图像内部的颜色冗余——尤其是颜色空间中的冗余。为此，我们提出数据集级颜色量化（DCQ），一种统一框架，通过减少颜色空间冗余来压缩视觉数据集，同时保留对模型训练至关重要的信息。DCQ通过在相似图像间强制一致的调色板表示、基于模型感知选择性保留语义重要颜色，并维持对有效特征学习必要的结构细节，实现压缩与保真平衡。在CIFAR-10、CIFAR-100、Tiny-ImageNet和ImageNet-1K上的大量实验表明，DCQ在极端压缩条件下显著提升训练性能，为数据集级存储压缩提供了可扩展且鲁棒的解决方案。
原文摘要 · Abstract (English)
Large-scale image datasets are fundamental to deep learning, but their high storage demands pose challenges for deployment in resource-constrained environments. While existing approaches reduce dataset size by discarding samples, they often ignore the significant redundancy within each image -- particularly in the color space. To address this, we propose Dataset Color Quantization (DCQ), a unified framework that compresses visual datasets by reducing color-space redundancy while preserving information crucial for model training. DCQ achieves this by enforcing consistent palette representations across similar images, selectively retaining semantically important colors guided by model perception, and maintaining structural details necessary for effective feature learning. Extensive experiments across CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet-1K show that DCQ significantly improves training performance under aggressive compression, offering a scalable and robust solution for dataset-level storage reduction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。