让大模型单次推理就能准确估算不确定性，速度更快。
Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation
- 用证据学习将多轮推理的教师模型压缩为单次推理学生模型
- 学生模型在分类任务中表现接近教师，且只需一次前向传播
- 适合需要快速可靠不确定性的实际部署场景
标准大模型在不确定性量化方面仍面临挑战，现有贝叶斯和集成方法需多次前向传播采样，计算开销大。本文提出一种证据知识蒸馏方法，将原本需多次前向传播的不确定性感知教师模型，蒸馏为仅需单次前向传播的学生模型，使用LoRA进行微调。比较了两种策略：传统softmax输出与基于狄利克雷分布的输出，后者能显式建模认知不确定性。实验表明，学生模型在分类任务上可达到与教师相当的预测与不确定性量化性能，且推理延迟显著降低。
原文摘要 · Abstract (English)
Accurate uncertainty quantification remains a key challenge for standard LLMs, prompting the adoption of Bayesian and ensemble-based methods. However, such methods typically necessitate computationally expensive sampling, involving multiple forward passes to effectively estimate predictive uncertainty. In this paper, we introduce an approach enabling uncertainty estimation in LLMs without incurring the heavy inference latency typically associated with sampling methods. Specifically, we distill uncertainty-aware teachers - originally requiring multiple forward passes - into single-pass students, fine-tuned using LoRA. We compare two distinct distillation strategies: one in which the student employs traditional softmax-based outputs, and another in which the student leverages Dirichlet-distributed outputs to explicitly model epistemic uncertainty via evidential learning. Empirical evaluation on classification tasks demonstrate that such students can achieve comparable predictive and uncertainty quantification performance relative to their teachers, while requiring only a single forward pass.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。