用小模型高效估算大模型的不确定性，降低幻觉风险。
Efficient Epistemic Uncertainty Estimation for Large Language Models via Knowledge Distillation
- 用多个小草稿模型计算分歧度，近似估计每词不确定性
- 在GSM8K上误差比基线低37%，接近复杂方法效果
- 推理开销极小，适合实际部署于安全关键任务
量化大型语言模型（LLMs）的不确定性对于缓解幻觉、实现安全关键任务中的风险感知部署至关重要。然而，通过深度集成（Deep Ensembles）估计认知不确定性（EU）在现代模型规模下计算成本过高。我们提出一种框架，利用小型草稿模型高效估算词级别欧氏不确定性，避免全规模集成。该方法基于偏差-方差分解理论，通过草稿间杰恩斯-申农散度（均值代理）和草稿混合与目标模型间的KL散度（偏差代理）近似欧盟。为进一步保证精度且不增加显著开销，引入在线随机蒸馏（OSD）以高效逼近目标聚合，并采用数据多样草稿策略（DDD）增强草稿多样性以更好逼近目标。在GSM8K上的大量实验表明，本方法相比基线将估计误差（RMSE）降低高达37%。关键的是，该方法在幻觉检测性能上可媲美如TokUR等复杂扰动方法，同时推理成本几乎可以忽略，为不确定性感知的LLM部署提供了实用方案。
原文摘要 · Abstract (English)
Quantifying uncertainty in Large Language Models (LLMs) is essential for mitigating hallucinations and enabling risk-aware deployment in safety-critical tasks. However, estimating Epistemic Uncertainty(EU) via Deep Ensembles is computationally prohibitive at the scale of modern models. We propose a framework that leverages the small draft models to efficiently estimate token-level EU, bypassing the need for full-scale ensembling. Theoretically grounded in a Bias-Variance Decomposition, our approach approximates EU via Jensen-Shannon divergence among drafts (variance proxy) and KL divergence between the draft mixture and the target (bias proxy). To further ensure accuracy without significant overhead, we introduce Online Stochastic Distillation (OSD) to efficiently approximate target aggregation and the Data-Diverse Drafts (DDD) strategy to enhance draft diversity for better target approximation. Extensive experiments on GSM8K demonstrate that our method reduces the estimation error (RMSE) by up to 37% compared to baselines. Crucially, our approach achieves Hallucination Detection performance competitive with heavy perturbation-based methods like TokUR while incurring negligible inference costs, offering a practical solution for uncertainty-aware LLM deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。