量化如何改变神经网络分类边界,用几何方法精准评估并优化。
Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers

- 用决策边界几何度量量化带来的变化,如位移、交点重排等。
- 4比特量化下准确率仍达97.33%,边界重合率达97.0%。
- 提出基于边界信息的校准停止策略,显著减少预测翻转。
本文通过局部对数边际半径、一阶边界位移、法向变化、切片边界交并比、网格预测变化、多类交点数量及低边际边界带翻转等指标,量化分析了量化对神经分类器决策边界的影响。在数字数据集上,8比特权重量化保持所有测试标签不变,主成分分析切片的边界掩码交并比为0.428;4比特时准确率为0.9733,边界交并比升至0.970,中位边界位移达0.0290。相邻量化层级间的插值揭示了多类交点处的可见重构,选定转换中出现12、34、17个三重交点单元。校准到测试的停止规则将数字集外部翻转率从0.0094降至0.0022,边界交并比从0.825降至0.524;该规则同样降低MNIST和Fashion-MNIST的翻转。在官方CIFAR-10子集上,按准确率选择的6比特PTQ-W产生0.0367的翻转率与0.184的边界交并比,而边界感知停止选择的8比特仅0.0083翻转率与0.048交并比。全量CIFAR-10三种子实验中,6比特PTQ-W相比浮点损失0.0029准确率,改变0.053的外部决策,24.5%的低边际边界带决策被修改。固定比特边界间隙正则项在4比特时使边界交并比从0.457降至0.435,边界对顺序翻转从0.3600降至0.3558,伴随精度代价;3比特压力测试暴露了该代理的调优极限。校准边界交并比在不同PTQ-W和优化舍入变体间,对预估外部边界交并比具有极强相关性(r=0.947–0.994)。
原文摘要 · Abstract (English)
We measured quantization-induced decision-boundary changes using local logit-margin radii, first-order boundary displacement, normal variation, slice-boundary Jaccard distance, grid prediction changes, multiclass junction counts, and low-margin boundary-band flips. On the digits benchmark, 8-bit weight quantization preserved all test labels while producing boundary-mask Jaccard \(0.428\) on the PCA slice; at 4 bits, accuracy remained \(0.9733\), while boundary Jaccard rose to \(0.970\) and median local boundary shift reached \(0.0290\). Interpolation between adjacent quantization levels localized the visible reconfigurations at multiclass junctions, with 12, 34, and 17 triple-junction cells in the selected transitions. Calibration-to-test stopping reduced the digits held-out flip rate from \(0.0094\) to \(0.0022\) and boundary Jaccard from \(0.825\) to \(0.524\); the same stopping rule also reduced flips on MNIST and Fashion-MNIST. On official CIFAR-10 subsets, PTQ-W selected by accuracy gave 6-bit flip \(0.0367\) and boundary Jaccard \(0.184\), whereas boundary-aware stopping selected 8-bit flip \(0.0083\) and boundary Jaccard \(0.048\). On full CIFAR-10 with three seeds, 6-bit PTQ-W lost \(0.0029\) accuracy relative to float, changed \(5.3\%\) of held-out decisions, and changed \(24.5\%\) of low-margin boundary-band decisions. A fixed-bit boundary-gap rounding term changed the trade-off at 4 bits by reducing boundary Jaccard from \(0.457\) to \(0.435\) and boundary-band pair-order flip from \(0.3600\) to \(0.3558\), with an accuracy trade-off; the 3-bit stress test exposed the tuning limit of this surrogate. Calibration boundary Jaccard predicted held-out boundary Jaccard across PTQ-W and optimized rounding variants with \(r=0.947\)--\(0.994\).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。