提出新审计方法,精准检测模型释放风险
A Deployment Audit of Release-Side Risk in Conformal Triage under Prevalence Shift

- 分离三类角色:校准、修正和评估,实现精准审计
- 发现低人工审查率可能遗漏阳性患者,存在释放风险
- 适合医疗决策系统部署前的风险评估者使用
共形分诊将预测得分转化为发布、紧急标记或转交人工审查的决策。当目标事件发生率发生变化时,仅关注覆盖率和人工审查率会遗漏未经过审查却发生目标事件的患者。为此,本文提出一种泄漏感知的发布侧共形分诊审计方法。该方法将目标样本分为三类不重叠角色:流行率校正、共形校准和保留的发布侧评估。此分离机制使审计能直接评估发布行为:有多少阳性患者被无审查释放,试点数据是否具备足够事件标签用于校准,以及发布-审查权衡如何变化。在非小细胞肺癌(NSCLC)回顾性队列上的应用表明,降低审查率可能具有误导性:经流行率校正后,聚合共形分支通过释放更多患者降低了审查率,但其中包含部分阳性患者。在审计框架内,按类别分支可作为稀缺性诊断工具:试点数据中事件标签过少,无法支持低审查率的释放规则。
原文摘要 · Abstract (English)
Conformal triage converts predictive scores into deployment actions that either release a case, flag it for urgent attention, or defer it to human review. Under an observed change in target-event prevalence, however, marginal coverage and human-review rate can miss whether patients who experience the target event are released without review. To address this gap, we introduce a leakage-aware deployment audit for release-side conformal triage. It first assigns target subjects to three non-overlapping roles: prevalence correction, conformal calibration, and held-out release-side evaluation. This separation then lets the audit evaluate release directly: how many event-positive patients are cleared without review, whether the pilot has enough event labels for calibration, and how the release-review trade-off shifts. Applying this audit to a retrospective non-small-cell lung cancer (NSCLC) target cohort shows why lower review can be misleading: after prevalence correction, the pooled conformal branch lowers review by releasing more patients, some of whom are event-positive. Within the audit, the classwise branch acts as a scarcity diagnostic: the pilot has too few event labels to support a low-review release rule.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。