用不确定性驱动路由,让量化模型又准又稳。
Uncertainty Makes It Stable: Curiosity-Driven Quantized Mixture-of-Experts
- 基于认知不确定性的路由机制,自动分配难样本给高精度专家。
- 4比特量化下准确率保留99.9%,能耗降低31%且延迟稳定。
- 适合对可靠性要求高的边缘设备,如医疗、自动驾驶场景。
在资源受限设备上部署深度神经网络面临两大挑战:在极端量化下保持精度,同时确保可预测的推理延迟。本文提出一种基于认知不确定性的量化混合专家框架(Curiosity-Driven Quantized Mixture-of-Experts),通过贝叶斯后验认知不确定性实现异构专家间的路由决策(BitNet 三值、1-16比特 BitLinear、训练后量化)。在音频分类基准(ESC-50、Quinn、UrbanSound8K)上,4比特量化仍保持全精度F1的99.9%(0.858 vs 0.859),压缩比达4倍,相比8比特节能31%,且两者与全精度无统计差异(p > 0.05)。关键在于,认知驱动路由同时提升准确率与稳定性:在Quinn数据集上F1从0.802升至0.809,跨折叠方差下降85%(p < 0.001,Levene检验),各数据集降幅达50%-94%。路由机制自组织:高精度8比特专家自动接收最不确定样本(置信度低20%,p < 0.001),轻量专家处理简单输入。基线方差已低的数据集未出现人为稳定性提升,证明机制针对真实认知不确定性而非过拟合。模型仅120万参数,提供可解释的精度感知路由,适用于对准确性与可预测性要求极高的安全敏感型边缘部署。
原文摘要 · Abstract (English)
Deploying deep neural networks on resource-constrained devices faces two critical challenges: maintaining accuracy under aggressive quantization while ensuring predictable inference latency. We present a curiosity-driven quantized Mixture-of-Experts framework that addresses both through Bayesian epistemic uncertainty-based routing across heterogeneous experts (BitNet ternary, 1-16 bit BitLinear, post-training quantization). Evaluated on audio classification benchmarks (ESC-50, Quinn, UrbanSound8K), our 4-bit quantization maintains 99.9 percent of full-precision F1 (0.858 vs 0.859) with 4x compression and 31 percent energy savings versus 8-bit, while both achieve statistical parity with full precision (p > 0.05). Crucially, curiosity-driven routing simultaneously improves accuracy and stability: on Quinn, F1 increases from 0.802 to 0.809 while cross-fold variance drops by 85 percent (p < 0.001, Levene's test), with reductions of 50 to 94 percent across datasets. The routing is self-organizing, with the high-precision 8-bit expert automatically receiving the most uncertain samples (20 percent lower confidence, p < 0.001), while lightweight experts handle easier inputs. Datasets with already low baseline variance show no artificial stability gain, confirming the mechanism targets genuine epistemic uncertainty rather than overfitting routing decisions. At 1.2M parameters, the framework provides interpretable, precision-aware routing suitable for safety-sensitive edge deployments where both accuracy and predictability are critical.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。