用图神经网络分析大模型多回答一致性，提升置信度估计准确性。
Graph-based Confidence Calibration for Large Language Models
- 构建响应一致性图，用GNN学习多个输出间的共识关系。
- 在多个基准数据集上显著改善置信度校准效果。
- 适合需要高可信度判断的医疗、金融等关键场景使用。
可靠的置信度估计对于提升大语言模型在高风险场景中的可信度至关重要。尽管重要，但准确估计大模型输出的置信度仍面临巨大挑战。本文提出一种辅助学习模型，基于大模型生成的多个输出之间的自一致性来评估回答正确性。方法构建一致性图以表征多个响应间的共识，并利用图神经网络（GNN）估计每个响应正确的概率。实验表明，该方法在多个基准数据集上表现优异，且对域外情况具有良好的泛化能力。
原文摘要 · Abstract (English)
Reliable confidence estimation is essential for enhancing the trustworthiness of large language models (LLMs), especially in high-stakes scenarios. Despite its importance, accurately estimating confidence in LLM responses remains a significant challenge. In this work, we propose using an auxiliary learning model to assess response correctness based on the self-consistency of multiple outputs generated by the LLM. Our method builds a consistency graph to represent the agreement among multiple responses and uses a graph neural network (GNN) to estimate the likelihood that each response is correct. Experiments demonstrate that this method has strong calibration performance on various benchmark datasets and generalizes well to out-of-domain cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。