让模型识别自己不懂的时刻,提升效率与可靠性。
When Models Know When They Do Not Know: Calibration, Cascading, and Cleaning
- 用校准后的置信度信号实现模型自我认知
- 可实现大模型与小模型级联,精度几乎无损
- 适合需要高可信度和数据清洗的场景
当模型能识别自身未知时,诸多可能性涌现。本文提出一种无需训练的通用方法,适用于视觉与语言模型,实现模型校准、级联与数据清洗。实证发现:单个模型中置信度越高,准确率越高;在验证集上校准的模型在保留测试集上仍保持校准状态。基于此,我们设计了两种应用:一是利用校准置信度进行优势路由,级联大模型与小模型,在几乎不损失精度的前提下提升效率;二是通过多模型集成与校准置信度,实现对ImageNet和MMLU数据集中标注错误样本的有效检测。结果表明,让模型识别自身无知是迈向更高效、可靠、可信AI的关键一步。
原文摘要 · Abstract (English)
When a model knows when it does not know, many possibilities emerge. The first question is how to enable a model to recognize that it does not know. A promising approach is to use confidence, computed from the model's internal signals, to reflect its ignorance. Prior work in specific domains has shown that calibration can provide reliable confidence estimates. In this work, we propose a simple, effective, and universal training-free method that applies to both vision and language models, performing model calibration, cascading, and data cleaning to better exploit a model's ability to recognize when it does not know. We first highlight two key empirical observations: higher confidence corresponds to higher accuracy within a single model, and models calibrated on the validation set remain calibrated on a held-out test set. These findings empirically establish the reliability and comparability of calibrated confidence. Building on this, we introduce two applications: (1) model cascading with calibrated advantage routing and (2) data cleaning based on model ensemble. Using the routing signal derived from the comparability of calibrated confidences, we cascade large and small models to improve efficiency with almost no compromise in accuracy, and we further cascade two models of comparable scale to achieve performance beyond either model alone. Leveraging multiple experts and their calibrated confidences, we design a simple yet effective data-cleaning method that balances precision and detection rate to identify mislabeled samples in ImageNet and Massive Multitask Language Understanding (MMLU) datasets. Our results demonstrate that enabling models to recognize when they do not know is a practical step toward more efficient, reliable, and trustworthy AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。