用集成方法量化大模型分类结果的不确定性，提升可信度。
Ensemble based approach to quantifying uncertainty of LLM based classifications
- 通过集成多个采样结果，捕捉模型输出的统计波动。
- 微调后模型对输入词汇变化的敏感度降低，输出更稳定。
- 适合需要判断模型置信度的高风险应用场景。
大型语言模型（LLMs）的输出取决于其内部参数和输入内容。本文提出假设：在贪婪采样策略下，模型输出的方差源于模型参数知识中的概念确定性以及输入的词汇差异。微调可降低模型输出对输入词汇变化的敏感性。该方法被应用于分类任务，提出一种概率方法来估计各类预测的确定性。
原文摘要 · Abstract (English)
The output of Large Language Models (LLMs) are a function of the internal model's parameters and the input provided into the context window. The hypothesis presented here is that under a greedy sampling strategy the variance in the LLM's output is a function of the conceptual certainty embedded in the model's parametric knowledge, as well as the lexical variance in the input. Finetuning the model results in reducing the sensitivity of the model output to the lexical input variations. This is then applied to a classification problem and a probabilistic method is proposed for estimating the certainties of the predicted classes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。