arXiv:2410.01767cs.LG2024-10ICLR被引 13

让预测结果更贴近实际决策需求,降低高风险场景下的错误成本。

Utility-Directed Conformal Prediction: A Decision-Aware Framework for Actionable Uncertainty Quantification

  • 基于校准预测构建决策感知的置信集,融合下游成本函数。
  • 在多个数据集上验证,相比传统方法显著降低决策成本。
  • 适用于医疗诊断等需结合领域知识的高风险决策场景。

决策导向的机器学习方法旨在训练模型时考虑其预测如何被用于后续优化问题,从而提升最终决策表现。然而,现有不确定性量化方法未纳入下游决策信息。本文提出一种基于校准预测的新方法,生成能反映下游成本函数的预测集,使其更适合作为高风险决策依据。该方法结合了校准方法的模块化、模型无关性及统计覆盖率保障,并引入用户指定的效用函数。理论证明该方法保持标准覆盖率。在多个数据集和效用指标上的实证评估表明,该方法显著低于标准校准方法的决策成本。一个真实医疗诊断案例显示,该方法有效利用皮肤科疾病的层级结构,生成具有临床意义的诊断集合,有助于皮肤病分诊,展示了如何将领域知识融入高风险决策过程。

原文摘要 · Abstract (English)

Interest has been growing in decision-focused machine learning methods which train models to account for how their predictions are used in downstream optimization problems. Doing so can often improve performance on subsequent decision problems. However, current methods for uncertainty quantification do not incorporate any information about downstream decisions. We develop a methodology based on conformal prediction to identify prediction sets that account for a downstream cost function, making them more appropriate to inform high-stakes decision-making. Our approach harnesses the strengths of conformal methods -- modularity, model-agnosticism, and statistical coverage guarantees -- while incorporating downstream decisions and user-specified utility functions. We prove that our methods retain standard coverage guarantees. Empirical evaluation across a range of datasets and utility metrics demonstrates that our methods achieve significantly lower costs than standard conformal methods. We present a real-world use case in healthcare diagnosis, where our method effectively incorporates the hierarchical structure of dermatological diseases. The method successfully generates sets with coherent diagnostic meaning, potentially aiding triage for dermatology diagnosis and illustrating how our method can ground high-stakes decision-making employing domain knowledge.

不确定性量化决策支持医疗诊断校准预测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。