用几何分散度量化生成模型不确定性，无需特定任务规则。
Directional Concentration Uncertainty: A representational approach to uncertainty quantification for generative models
- 基于vMF分布测量生成结果的嵌入分散度，捕捉不确定性。
- 在多模态任务中表现优于或相当语义熵方法，校准精度高。
- 适用于多模态与智能体框架，通用性强，适合部署应用。
在提升生成模型可信度与鲁棒性的关键任务中，不确定性量化（UQ）方法展现出令人鼓舞的潜力。然而，许多现有方法依赖僵化的启发式规则，难以跨任务和模态泛化。本文提出一种新型灵活的UQ框架——方向集中不确定性（DCU），基于冯·米塞斯-费舍尔（vMF）分布，通过连续嵌入测量语言模型生成输出的几何分散度，实现无任务特异性启发式的不确定性量化。实验表明，DCU在多模态复杂任务中达到或超越先前方法（如语义熵，Kuhn et al., 2023）的校准水平，具备良好泛化能力。本文还展示了DCU的广泛应用前景及其在多模态与智能体系统中集成的潜力。
原文摘要 · Abstract (English)
In the critical task of making generative models trustworthy and robust, methods for Uncertainty Quantification (UQ) have begun to show encouraging potential. However, many of these methods rely on rigid heuristics that fail to generalize across tasks and modalities. Here, we propose a novel framework for UQ that is highly flexible and approaches or surpasses the performance of prior heuristic methods. We introduce Directional Concentration Uncertainty (DCU), a novel statistical procedure for quantifying the concentration of embeddings based on the von Mises-Fisher (vMF) distribution. Our method captures uncertainty by measuring the geometric dispersion of multiple generated outputs from a language model using continuous embeddings of the generated outputs without any task specific heuristics. In our experiments, we show that DCU matches or exceeds calibration levels of prior works like semantic entropy (Kuhn et al., 2023) and also generalizes well to more complex tasks in multi-modal domains. We present a framework for the wider potential of DCU and its implications for integration into UQ for multi-modal and agentic frameworks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。