用云端模型指导边缘设备,让低复杂度模型也能可靠输出可信预测范围。
Distilling Calibration via Conformalized Credal Inference
- 从高复杂度云端模型提取校准信息,离线生成阈值用于边缘推理
- 在100%置信度下保证边缘模型预测落在可信区间内,显著提升校准性
- 适合资源受限的边缘AI场景,尤其需可靠不确定性的应用
将人工智能模型部署于边缘设备时,需在严格算力与内存限制下保障关键决策的可靠性。传统基于贝叶斯推断的不确定性量化需维护多个模型集成,超出边缘设备计算能力。本文提出一种低复杂度方法:在离线阶段,利用云端高复杂度模型生成的预测概率,确定云-边模型间典型差异的阈值;运行时,该阈值用于构造可信集——即以用户指定置信水平(如95%)保证包含云端模型预测结果的概率区间。可信集通过在预测概率单纯形上对发散度量进行阈值化获得。视觉与语言任务实验表明,所提方法Conformalized Distillation for Credal Inference (CD-CI) 在校准性能上显著优于拉普拉斯近似等低复杂度贝叶斯方法,是边缘AI部署中高效可靠的解决方案。
原文摘要 · Abstract (English)
Deploying artificial intelligence (AI) models on edge devices involves a delicate balance between meeting stringent complexity constraints, such as limited memory and energy resources, and ensuring reliable performance in sensitive decision-making tasks. One way to enhance reliability is through uncertainty quantification via Bayesian inference. This approach, however, typically necessitates maintaining and running multiple models in an ensemble, which may exceed the computational limits of edge devices. This paper introduces a low-complexity methodology to address this challenge by distilling calibration information from a more complex model. In an offline phase, predictive probabilities generated by a high-complexity cloud-based model are leveraged to determine a threshold based on the typical divergence between the cloud and edge models. At run time, this threshold is used to construct credal sets -- ranges of predictive probabilities that are guaranteed, with a user-selected confidence level, to include the predictions of the cloud model. The credal sets are obtained through thresholding of a divergence measure in the simplex of predictive probabilities. Experiments on visual and language tasks demonstrate that the proposed approach, termed Conformalized Distillation for Credal Inference (CD-CI), significantly improves calibration performance compared to low-complexity Bayesian methods, such as Laplace approximation, making it a practical and efficient solution for edge AI deployments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。