用随机森林预测电镜图像检测模型的性能,快速评估可靠性。
Predicting Performance of Object Detection Models in Electron Microscopy Using Random Forests
- 基于检测结果提取特征,训练随机森林预测F1分数
- 测试集上预测误差仅0.09,解释力达77%
- 适用于不同材料与成像条件,助用户判断是否需微调
在应用机器学习于新未标注数据集时,量化预测不确定性至关重要。本文提出一种方法,通过随机森林回归模型预测基于深度学习的物体检测模型在透射电子显微镜(TEM)图像中检测缺陷的性能,重点针对金属合金中辐照诱导空洞的检测。该模型利用待评估检测模型预测结果提取特征,快速预测其F1分数。在测试数据上,平均绝对误差(MAE)为0.09,$R^2$得分为0.77,表明预测值与真实F1分数具有显著相关性。该方法在三个不同成像与材料领域的TEM数据集上均表现稳健。该方法可帮助用户评估模型在特定数据集上的可靠性,识别潜在领域偏移,判断是否需要微调或补充训练数据以最大化模型效能。
原文摘要 · Abstract (English)
Quantifying prediction uncertainty when applying object detection models to new, unlabeled datasets is critical in applied machine learning. This study introduces an approach to estimate the performance of deep learning-based object detection models for quantifying defects in transmission electron microscopy (TEM) images, focusing on detecting irradiation-induced cavities in TEM images of metal alloys. We developed a random forest regression model that predicts the object detection F1 score, a statistical metric used to evaluate the ability to accurately locate and classify objects of interest. The random forest model uses features extracted from the predictions of the object detection model whose uncertainty is being quantified, enabling fast prediction on new, unlabeled images. The mean absolute error (MAE) for predicting F1 of the trained model on test data is 0.09, and the $R^2$ score is 0.77, indicating there is a significant correlation between the random forest regression model predicted and true defect detection F1 scores. The approach is shown to be robust across three distinct TEM image datasets with varying imaging and material domains. Our approach enables users to estimate the reliability of a defect detection and segmentation model predictions and assess the applicability of the model to their specific datasets, providing valuable information about possible domain shifts and whether the model needs to be fine-tuned or trained on additional data to be maximally effective for the desired use case.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。