提出混合量化方法PTQAT,用更少计算量实现3D感知模型更高精度。
PTQAT: A Hybrid Parameter-Efficient Quantization Algorithm for 3D Perception Tasks
- 选关键层做微调,其余层用快速量化,平衡速度与精度。
- 在nuScenes上比纯QAT提升0.2%-1.0%检测与分割指标。
- 支持多种模型结构和4比特量化,适合部署优化场景。
后训练量化(PTQ)和量化感知训练(QAT)是主流量化方法。但PTQ常导致性能大幅下降,而QAT需大量显存和训练时间。本文提出PTQAT,一种通用的混合量化算法,用于高效部署3D感知网络。为解决PTQ与QAT的速度-精度权衡,该方法选择关键层进行QAT微调,其余层采用PTQ。出乎意料的是,对量化前后输出差异较小的层进行微调,反而能更大程度提升量化精度,因能更好补偿误差传播。相比全量微调,PTQAT冻结近50%可量化层,仍达到接近QAT的性能。该方法支持4比特及以下量化,适用于CNN与Transformer等多类架构。在nuScenes数据集上的多项3D感知任务(目标检测、语义分割、占据预测)实验表明,其性能持续优于纯QAT基线:目标检测提升0.2%-0.9% NDS与0.3%-1.0% mAP,语义分割与占据预测提升0.3%-2.0% mIoU,同时微调权重更少。
原文摘要 · Abstract (English)
Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) represent two mainstream model quantization approaches. However, PTQ often leads to unacceptable performance degradation in quantized models, while QAT imposes substantial GPU memory requirements and extended training time due to weight fine-tuning. In this paper, we propose PTQAT, a novel general hybrid quantization algorithm for the efficient deployment of 3D perception networks. To address the speed accuracy trade-off between PTQ and QAT, our method selects critical layers for QAT fine-tuning and performs PTQ on the remaining layers. Contrary to intuition, fine-tuning the layers with smaller output discrepancies before and after quantization, rather than those with larger discrepancies, actually leads to greater improvements in the model's quantization accuracy. This means we better compensate for quantization errors during their propagation, rather than addressing them at the point where they occur. The proposed PTQAT achieves similar performance to QAT with more efficiency by freezing nearly 50% of quantifiable layers. Additionally, PTQAT is a universal quantization method that supports various quantization bit widths (4 bits) as well as different model architectures, including CNNs and Transformers. The experimental results on nuScenes across diverse 3D perception tasks, including object detection, semantic segmentation, and occupancy prediction, show that our method consistently outperforms QAT-only baselines. Notably, it achieves 0.2%-0.9% NDS and 0.3%-1.0% mAP gains in object detection, 0.3%-2.0% mIoU gains in semantic segmentation and occupancy prediction while fine-tuning fewer weights.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。