arXiv:2604.08885cs.LG2026-04被引 1

给大模型加不确定性分析,让预测结果更可信。

Uncertainty-Aware Transformers: Conformal Prediction for Language Models

论文配图:Uncertainty-Aware Transformers: Conformal Prediction for Language Models
图 1 · 摘自论文原文
  • 用校准预测法分析Transformer内部表示,生成带置信度的预测集。
  • 在BERT-tiny上提升准确率4.09%,预测集更小且更可靠。
  • 适合对可靠性要求高的场景,如医疗、金融等关键任务。

Transformer在人工智能领域影响深远,尤其在大语言模型及其变体中。然而,如同神经网络一样,其黑箱特性限制了在高风险场景中的信任与部署。为使模型在关键应用中真正有用且可信,不仅需提供预测,还需阐明决策依据。本文提出针对基于Transformer的语言模型的不确定性量化框架CONFIDE(CONformal prediction for FIne-tuned DEep language models)。该框架将校准预测应用于编码器类架构(如BERT、RoBERTa)的内部嵌入表示,支持超参数调优。CONFIDE使用[CLS]标记嵌入或展平的隐藏状态构建类别条件非一致性得分,实现具有实例级解释性的统计有效预测集。实验表明,CONFIDE在BERT-tiny上最高可提升测试准确率4.09%,并优于NM2和VanillaNN等先前方法,在正确性效率(即包含真实标签时预测集的期望大小)方面表现更佳。研究发现,早期和中间层的Transformer表示通常更适合校准预测,具有更好的校准性和语义意义。在资源受限模型和标签模糊的高风险任务中,CONFIDE在鲁棒性和可解释性方面优于基于Softmax的不确定性方法。本文将CONFIDE定位为一种实用框架,用于诊断及提升现有校准基线的效率与鲁棒性。

原文摘要 · Abstract (English)

Transformers have had a profound impact on the field of artificial intelligence, especially on large language models and their variants. However, as was the case with neural networks, their black-box nature limits trust and deployment in high-stakes settings. For models to be genuinely useful and trustworthy in critical applications, they must provide more than just predictions: they must supply users with a clear understanding of the reasoning that underpins their decisions. This article presents an uncertainty quantification framework for transformer-based language models. This framework, called CONFIDE (CONformal prediction for FIne-tuned DEep language models), applies conformal prediction to the internal embeddings of encoder-only architectures, like BERT and RoBERTa, while enabling hyperparameter tuning. CONFIDE uses either [CLS] token embeddings or flattened hidden states to construct class-conditional nonconformity scores, enabling statistically valid prediction sets with instance-level explanations. Empirically, CONFIDE improves test accuracy by up to 4.09% on BERT-tiny and achieves greater correct efficiency (i.e., the expected size of the prediction set conditioned on it containing the true label) compared to prior methods, including NM2 and VanillaNN. We show that early and intermediate transformer layers often yield better-calibrated and more semantically meaningful representations for conformal prediction. In resource-constrained models and high-stakes tasks with ambiguous labels, CONFIDE offers robustness and interpretability where softmax-based uncertainty fails. We position CONFIDE as a framework for practical diagnostic and efficiency/robustness improvement over prior conformal baselines.

不确定性语言模型校准预测可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。