通过校准置信度评估语言模型关系知识，发现多数模型过于自信。
Self-Aware Knowledge Probing: Evaluating Language Models' Relational Knowledge through Confidence Calibration
- 引入三种置信度校准机制：内在信心、结构一致性和语义基础性。
- 六种掩码模型和十种因果模型均存在过度自信问题，尤其掩码模型更严重。
- 重述句子导致的不一致性更能反映真实置信度，适合评估模型可靠性。
知识探针用于量化语言模型（LM）在预训练过程中习得的关系知识。现有探针主要依赖预测准确率和精确率等指标，但未考虑模型置信度的可靠性，即置信分数的校准程度。本文提出一种新的关系知识校准探针框架，涵盖三种模型置信度模态：(1) 内在置信度，(2) 结构一致性，(3) 语义基础性。对十种因果模型和六种掩码模型的广泛分析表明，大多数模型尤其是以掩码为目标预训练的模型存在过度自信现象。表现最佳的置信度估计来自能捕捉陈述重述引发不一致性的方法。此外，即使最大规模的预训练模型也未能准确编码语言中置信表达的语义。
原文摘要 · Abstract (English)
Knowledge probing quantifies how much relational knowledge a language model (LM) has acquired during pre-training. Existing knowledge probes evaluate model capabilities through metrics like prediction accuracy and precision. Such evaluations fail to account for the model's reliability, reflected in the calibration of its confidence scores. In this paper, we propose a novel calibration probing framework for relational knowledge, covering three modalities of model confidence: (1) intrinsic confidence, (2) structural consistency and (3) semantic grounding. Our extensive analysis of ten causal and six masked language models reveals that most models, especially those pre-trained with the masking objective, are overconfident. The best-calibrated scores come from confidence estimates that account for inconsistencies due to statement rephrasing. Moreover, even the largest pre-trained models fail to encode the semantics of linguistic confidence expressions accurately.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。