用临床需求定制优化指标，提升医疗模型实际表现
Beyond validation loss: Clinically-tailored optimization metrics improve a model's clinical performance
- 用临床相关指标替代传统验证损失进行模型优化
- 实验显示新方法在临床任务上表现更优
- 适合关注医疗模型落地效果的研究者
机器学习模型优化通常依赖验证损失，但医疗领域目标不同于传统机器学习：模型需满足特定临床需求，而非仅最小化训练损失。本文通过两个受控实验表明，使用针对临床需求定制的评估指标进行优化，能显著提升模型在真实临床任务中的表现。尽管定义和实现这些指标需额外工作，但能更准确地推动模型向临床可用性靠拢。该方法不依赖可导性，适用范围更广，有助于实现医疗AI的核心目标——真正改善临床实践。
原文摘要 · Abstract (English)
A key task in ML is to optimize models at various stages, e.g. by choosing hyperparameters or picking a stopping point. A traditional ML approach is to use validation loss, i.e. to apply the training loss function on a validation set to guide these optimizations. However, ML for healthcare has a distinct goal from traditional ML: Models must perform well relative to specific clinical requirements, vs. relative to the loss function used for training. These clinical requirements can be captured more precisely by tailored metrics. Since many optimization tasks do not require the driving metric to be differentiable, they allow a wider range of options, including the use of metrics tailored to be clinically-relevant. In this paper we describe two controlled experiments which show how the use of clinically-tailored metrics provide superior model optimization compared to validation loss, in the sense of better performance on the clinical task. The use of clinically-relevant metrics for optimization entails some extra effort, to define the metrics and to code them into the pipeline. But it can yield models that better meet the central goal of ML for healthcare: strong performance in the clinic.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。