让模型在不同边缘设备上低比特部署更稳定，无需针对厂商调整
Quant-Trim in Practice: Improved Cross-Platform Low-Bit Deployment on Edge NPUs
- 训练时用渐进式伪量化和反向剪枝，对齐整数计算网格
- 跨平台部署准确率提升，减少对编译器调参依赖
- 兼容各种量化方式，不需改模型结构或适配厂商
专用边缘加速器依赖低比特量化，但不同厂商编译器在缩放、裁剪和内核支持上差异大，常作为黑盒处理。同一浮点检查点在不同后端可能产生不一致的精度，迫使实践者调整参数或重构模型以适配厂商支持的操作子。我们提出Quant-Trim，一种训练阶段方法，生成对后端和精度选择鲁棒的硬件中立检查点。它结合渐进式伪量化以对齐训练与部署整数网格，以及反向剪枝以抑制异常值导致的尺度膨胀，同时保持可学习性。Quant-Trim与量化方案无关（对称/非对称、按张量/按通道、INT8/INT4），无需厂商特定图修改。在多种模型与任务上，该方法缩小了浮点到低比特的差距，降低对编译器启发式或校准的依赖，并避免每后端重训。我们报告了静态/动态激活缩放及不同操作覆盖下的精度、延迟、吞吐、每次推理能耗与成本。
原文摘要 · Abstract (English)
Specialized edge accelerators rely on low-bit quantization, but vendor compilers differ in scaling, clipping, and kernel support, often as black boxes. The same floating-point (FP) checkpoint can therefore yield inconsistent accuracy across backends, forcing practitioners to tweak flags or refactor models to vendor-friendly operator subsets. We introduce Quant-Trim, a training-phase method that produces a hardware-neutral checkpoint robust to backend and precision choices. It combines progressive fake quantization to align training with the deployed integer grid and reverse pruning to tame outlier-driven scale inflation while preserving learnability. Quant-Trim is agnostic to quantization schemes (symmetric/asymmetric, per-tensor/per-channel, INT8/INT4) and requires no vendor-specific graph changes. Across models and tasks, it narrows the FP-to-low-bit gap, reduces dependence on compiler heuristics/calibration, and avoids per-backend retraining. We report accuracy and edge metrics latency, throughput, energy per inference, and cost under static/dynamic activation scaling and varying operator coverage.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。