通过聚类语义一致性评估大模型幻觉，提升回答可信度判断能力。
Cleanse: Uncertainty Estimation Approach Using Clustering-based Semantic Consistency in LLMs
- 基于隐藏层嵌入聚类，计算内部一致性占比来量化不确定性。
- 在SQuAD和CoQA上验证，能有效识别LLaMA和Mistral系列模型的幻觉输出。
- 适合关注大模型安全性和可靠性评估的研究者与开发者使用。
尽管大语言模型（LLMs）在各类自然语言任务中表现优异，但其生成不准确内容的幻觉问题仍是关键挑战，直接影响模型的安全性与可靠性。不确定性估计被广泛用于衡量生成结果的幻觉程度，以区分正确与错误回答。本文提出一种有效的不确定性估计方法——基于聚类的语义一致性（Cleanse）。Cleanse通过聚类分析包含充分生成语义信息的模型隐藏层嵌入，以内部一致性占总一致性的比例来量化不确定性。该方法在四个现成模型（LLaMA-7B、LLaMA-13B、LLaMA2-7B、Mistral-7B）和两个问答基准（SQuAD、CoQA）上进行了验证，证明其对幻觉检测具有显著有效性。
原文摘要 · Abstract (English)
Despite the outstanding performance of large language models (LLMs) across various NLP tasks, hallucinations in LLMs--where LLMs generate inaccurate responses--remains as a critical problem as it can be directly connected to a crisis of building safe and reliable LLMs. Uncertainty estimation is primarily used to measure hallucination levels in LLM responses so that correct and incorrect answers can be distinguished clearly. This study proposes an effective uncertainty estimation approach, \textbf{Cl}ust\textbf{e}ring-based sem\textbf{an}tic con\textbf{s}ist\textbf{e}ncy (\textbf{Cleanse}). Cleanse quantifies the uncertainty with the proportion of the intra-cluster consistency in the total consistency between LLM hidden embeddings which contain adequate semantic information of generations, by employing clustering. The effectiveness of Cleanse for detecting hallucination is validated using four off-the-shelf models, LLaMA-7B, LLaMA-13B, LLaMA2-7B and Mistral-7B and two question-answering benchmarks, SQuAD and CoQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。