提出概率框架，让模型按输入动态调整量化参数。
A probabilistic framework for dynamic quantization
- 用轻量代理模型建模前激活值分布，实现输入自适应量化
- 在多个视觉任务上性能损失可忽略，计算开销极低
- 适合部署时需兼顾精度与效率的边缘设备场景
我们提出一种概率框架，用于神经网络的动态量化，可高效实现输入自适应的量化参数重缩放。该框架通过轻量级代理模型对网络前激活值进行概率建模，能够在不显著增加内存开销的前提下，基于每条输入动态调整量化参数。我们在一系列主流计算机视觉任务和模型上验证了该方法，观察到性能损失几乎可以忽略。相比标准量化策略，该方法在性能与计算开销之间取得了最佳平衡。
原文摘要 · Abstract (English)
We propose a probabilistic framework for dynamic quantization of neural networks that allows for a computationally efficient input-adaptive rescaling of the quantization parameters. Our framework applies a probabilistic model to the network's pre-activations through a lightweight surrogate, enabling the adaptive adjustment of the quantization parameters on a per-input basis without significant memory overhead. We validate our approach on a set of popular computer vision tasks and models, observing only a negligible loss in performance. Our method strikes the best performance and computational overhead tradeoff compared to standard quantization strategies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。