提出新方法生成适配硬件量化数据,显著提升零样本量化准确率
Data Generation for Hardware-Friendly Post-Training Quantization
- 联合优化全部生成图像,突破内存限制
- 引入增强预处理和分布拉伸损失,减少真实与合成数据差异
- 在分类与检测任务中准确率提升最高达30%,媲美真实数据
在隐私与安全约束下,基于合成数据的零样本量化(ZSQ)是后训练量化(PTQ)的关键方法。然而,现有数据生成方法难以有效生成适用于全模型层量化(即硬件友好量化)的合成数据。我们分析基于批量归一化(BN)匹配的生成方法,发现三个关键差距:1)现有算法未全局优化整个合成数据集;2)训练时的数据增强常被忽略;3)因末层缺乏BN导致特征分布发生偏移。这些差距严重影响了硬件友好量化下的ZSQ性能。本文提出针对硬件量化的数据生成方法(DGH),通过联合优化所有生成图像克服内存限制;设计预处理阶段模拟数据增强并结合自然图像先验提升图像质量;提出新型分布拉伸损失,对齐真实与合成数据在模型输出层的特征分布支持范围,可适配多种任务。DGH在多个任务中显著提升量化性能,在分类与目标检测中实现高达30%的准确率提升,通常达到与真实数据相当的水平。
原文摘要 · Abstract (English)
Zero-shot quantization (ZSQ) using synthetic data is a key approach for post-training quantization (PTQ) under privacy and security constraints. However, existing data generation methods often struggle to effectively generate data suitable for hardware-friendly quantization, where all model layers are quantized. We analyze existing data generation methods based on batch normalization (BN) matching and identify several gaps between synthetic and real data: 1) Current generation algorithms do not optimize the entire synthetic dataset simultaneously; 2) Data augmentations applied during training are often overlooked; and 3) A distribution shift occurs in the final model layers due to the absence of BN in those layers. These gaps negatively impact ZSQ performance, particularly in hardware-friendly quantization scenarios. In this work, we propose Data Generation for Hardware-friendly quantization (DGH), a novel method that addresses these gaps. DGH jointly optimizes all generated images, regardless of the image set size or GPU memory constraints. To address data augmentation mismatches, DGH includes a preprocessing stage that mimics the augmentation process and enhances image quality by incorporating natural image priors. Finally, we propose a new distribution-stretching loss that aligns the support of the feature map distribution between real and synthetic data. This loss is applied to the model's output and can be adapted to various tasks. DGH demonstrates significant improvements in quantization performance across multiple tasks, achieving up to a 30% increase in accuracy for hardware-friendly ZSQ in both classification and object detection, often performing on par with real data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。