AI医疗干预需权衡服务容量与用户响应噪声,优化阈值可显著提升效果。
Deployment of AI-Assisted Interventions: Capacity Constraints and Noisy Compliance

- 基于服务容量和用户行为不确定性,动态调整干预阈值以平衡资源利用率与高价值个体覆盖。
- 传统仅追求预测准确率的算法选择方式在实际中效果不佳,可能浪费服务能力或错失关键患者。
- 提出新指标OpAUC,能更准确评估算法在真实场景中的表现,适合医疗、教育等资源有限领域。
AI工具正广泛应用于医疗、教育和招聘中的精准干预。算法对个体评分,对高于阈值者触发联络,鼓励其申请服务,再由服务提供方响应申请。现行做法通常通过最大化预测准确性来设定阈值并选择算法,假设更高预测精度必然带来更好结果。然而,在服务容量有限且个体响应具有随机性的条件下,此方法并非最优。此时最优阈值需权衡两方面:确保全部服务能力被充分利用(利用率),以及在申请竞争下仍能优先服务高价值个体(避免资源被低价值请求挤占)。本文刻画了最优阈值特性,并证明仅依赖预测准确性的策略普遍次优。此外,由于最优阈值随服务容量变化,传统算法评估指标如AUC对所有阈值一视同仁,与实际运营表现脱节。为此提出新指标——操作性AUC(OpAUC),证明其能引导出最优算法选择。最后,基于脓毒症早期预警数据的案例研究显示,改进阈值与算法选择可带来显著效果提升。
原文摘要 · Abstract (English)
AI tools increasingly guide targeted interventions in healthcare, education, and recruiting. Algorithms score individuals, trigger outreach to those above a threshold (e.g., high-risk or high-value), and encourage them to request service; then providers deliver service to those who request. Standard practice sets the threshold and selects the algorithm to maximize predictive accuracy, assuming that better predictions yield better outcomes. We show that this approach is suboptimal when limited service capacity and probabilistic behavioral responses influence who receives service. In such settings, the optimal score threshold must balance two effects: ensuring all capacity is filled (utilization) and ensuring high-value individuals are served despite competition between requests (cannibalization). We characterize the optimal threshold and prove that policies based solely on predictive accuracy are generally suboptimal. Further, because optimal thresholds vary with service capacity, algorithm selection metrics like AUC, which weight all thresholds equally, are misaligned with operational performance. We introduce a new metric--Operational AUC (OpAUC)--and show it leads to optimal algorithm selection. Finally, we conduct a case study on sepsis early warning data and illustrate the magnitude of improvement that can be achieved from improved threshold and algorithm selection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。