为机器学习分类模型的不确定性评估建立计量学框架。
A metrological framework for uncertainty evaluation in machine learning classification models
- 基于概率质量函数和统计量构建不确定性评估框架。
- 可适用于气候观测与医疗诊断等高影响场景的分类模型。
- 填补了国际计量标准对类别变量不确定性的空白，适合相关领域研究者。
机器学习（ML）分类模型在气候与地球观测、医学诊断及生物气溶胶监测等应用中日益重要，其预测结果需附带不确定性信息。然而，根据国际计量学词汇（VIM），ML分类输出属于名义属性，而该属性的不确定性评估概念在VIM中未定义，且《测量不确定度表达指南》（GUM）也未涵盖此类评估。本文提出一种针对名义属性的计量学不确定性评估框架，基于概率质量函数及其统计量，适用于机器学习分类模型。通过气候与地球观测、医学诊断两个典型应用案例，展示了该框架的实际适用性与社会价值。本框架有望扩展GUM以涵盖名义属性的不确定性，使二者均能应用于机器学习分类模型。
原文摘要 · Abstract (English)
Machine learning (ML) classification models are increasingly being used in a wide range of applications where it is important that predictions are accompanied by uncertainties, including in climate and earth observation, medical diagnosis and bioaerosol monitoring. The output of an ML classification model is a type of categorical variable known as a nominal property in the International Vocabulary of Metrology (VIM). However, concepts related to uncertainty evaluation for nominal properties are not defined in the VIM, nor is such evaluation addressed by the Guide to the Expression of Uncertainty in Measurement (GUM). In this paper we propose a metrological conceptual uncertainty evaluation framework for nominal properties. This framework is based on probability mass functions and summary statistics thereof, and it is applicable to ML classification. We also illustrate its use in the context of two applications that exemplify the issues and have significant societal impact, namely, climate and earth observation and medical diagnosis. Our framework would enable an extension of the GUM to uncertainty for nominal properties, which would make both applicable to ML classification models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。