提出可选量化框架，让ONNX模型在不同硬件上高效部署且精度损失小。
A Selective Quantization Tuner for ONNX Models
- 基于性能分析与多目标优化，智能选择部分层量化以平衡效率与精度。
- 相比全量化模型，精度损失降低54.14%，模型大小仍减少98.18%。
- 适用于资源受限设备，支持跨CPU/GPU部署，适合边缘计算场景。
量化通过降低深度神经网络的精度来减小模型尺寸和计算开销，但常伴随精度下降。全量化模型可能造成显著精度损失，且资源受限的硬件加速器可能不支持所有量化操作。一种常见方法是选择性量化，仅对部分层进行量化，其余保持全精度。然而，如何在精度与效率间找到最佳平衡仍是挑战。为此，我们提出SeQTO框架，支持ONNX模型在多种CPU和GPU设备上的选择性量化、部署与执行，结合性能分析与多目标优化。SeQTO生成选择性量化模型，在不同硬件上部署并评估准确率与模型大小等指标，采用帕累托前沿优化法筛选最优方案，并提供可视化结果。我们在四个ONNX模型上，于两种量化设置下，对CPU与GPU设备进行了评估。结果表明，SeQTO能有效识别高质量的选择性量化模型，相较全量化模型，精度损失最多降低54.14%，同时保持最高达98.18%的模型压缩率。
原文摘要 · Abstract (English)
Quantization reduces the precision of deep neural networks to lower model size and computational demands, but often at the expense of accuracy. Fully quantized models can suffer significant accuracy degradation, and resource-constrained hardware accelerators may not support all quantized operations. A common workaround is selective quantization, where only some layers are quantized while others remain at full precision. However, determining the optimal balance between accuracy and efficiency is a challenging task. To this direction, we propose SeQTO, a framework that enables selective quantization, deployment, and execution of ONNX models on diverse CPU and GPU devices, combined with profiling and multi-objective optimization. SeQTO generates selectively quantized models, deploys them across hardware accelerators, evaluates performance on metrics such as accuracy and size, applies Pareto Front-based objective minimization to identify optimal candidates, and provides visualization of results. We evaluated SeQTO on four ONNX models under two quantization settings across CPU and GPU devices. Our results show that SeQTO effectively identifies high-quality selectively quantized models, achieving up to 54.14% lower accuracy loss while maintaining up to 98.18% of size reduction compared to fully quantized models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。