arXiv:2509.24608cs.LGstat.ML2025-09

比较决策曲线与成本曲线,揭示其内在联系及适用场景。

Evaluating classification performance across operating contexts: A comparison of decision curve analysis and cost curves

  • 基于校准分数和正负样本价值比设定阈值
  • 净收益与贝叶斯损失在任意阈值下选优一致
  • 贝叶斯曲线更通用,面积即为贝叶斯得分

分类模型通常输出得分并用决策阈值生成分类结果。恰当的模型评估需考虑实际使用场景中正确分类正负样本的相对价值,这会影响最优阈值选择。决策曲线分析(DCA)和成本曲线分别评估预测模型在不同阈值下的期望效用和期望损失。本文比较了DCA与成本曲线的关系及其优劣。结果表明,决策曲线与一种特定成本曲线——贝叶斯曲线密切相关:二者均假设模型得分已校准,且阈值由正负样本正确分类的相对价值决定,横轴等价。净收益(DCA使用)与贝叶斯损失(贝叶斯曲线使用)在任一阈值下总选择同一最优模型;跨阈值时,贝叶斯损失差异可直接比较,而净收益差异不可比。贝叶斯曲线适用范围更广(当更多阈值合理时),其曲线下面积即为贝叶斯评分。本文还提出,两类图中均可加入参考线,并建议使用上包络决策曲线作为DCA的对比基准,展示仅通过再校准可能获得的净收益提升。

原文摘要 · Abstract (English)

Classification models typically predict a score and use a decision threshold to produce a classification. Appropriate model evaluation should carefully consider the context in which a model will be used, including the relative value of correct classifications of positive versus negative examples, which affects the threshold that should be used. Decision curve analysis (DCA) and cost curves are model evaluation approaches that assess the expected utility and expected loss of prediction models, respectively, across decision thresholds. We compared DCA and cost curves to determine how they are related, and their strengths and limitations. We demonstrate that decision curves are closely related to a specific type of cost curve called a Brier curve. Both curves are derived assuming model scores are calibrated and setting the classification threshold using the relative value of correct positive and negative classifications, and the x-axis of both curves are equivalent. Net benefit (used for DCA) and Brier loss (used for Brier curves) will always choose the same model as optimal at any given threshold. Across thresholds, differences in Brier loss are comparable whereas differences in net benefit cannot be compared. Brier curves are more generally applicable (when a wider range of thresholds are plausible), and the area under the Brier curve is the Brier score. We demonstrate that reference lines common in each space can be included in either and suggest the upper envelope decision curve as a useful comparison for DCA showing the possible gain in net benefit that could be achieved through recalibration alone.

模型评估决策曲线成本曲线

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。