解决临床NLP模型时间泄漏问题,提升部署安全性。
Building Safe and Deployable Clinical Natural Language Processing under Temporal Leakage Constraints
- 引入轻量级审计流程,在训练前识别并抑制时间泄漏信号。
- 审计后模型预测更保守,校准性更好,减少对出院关键词依赖。
- 适合关注临床模型安全与可部署性的研究者和医疗AI开发者。
临床自然语言处理(NLP)模型在支持医院出院规划方面展现出潜力,但基于病历的模型易受时间与词汇泄漏影响,文档中的未来临床决策痕迹会夸大预测性能。这在真实部署中可能引发严重风险,导致过度自信或时间上无效的预测,干扰临床流程并危及患者安全。本研究聚焦于在时间泄漏约束下构建安全且可部署的临床NLP系统所需的整体设计选择。提出一种轻量级审计管道,将可解释性融入模型开发过程,以识别并抑制训练前的泄漏敏感信号。以择期脊柱手术后次日出院预测为案例,评估审计对预测行为、校准性和安全相关权衡的影响。结果显示,经审计的模型具有更保守且更优校准的概率估计,对出院相关词汇线索的依赖显著降低。研究强调,可部署的临床NLP系统应优先考虑时间有效性、校准性和行为鲁棒性,而非乐观的性能表现。
原文摘要 · Abstract (English)
Clinical natural language processing (NLP) models have shown promise for supporting hospital discharge planning by leveraging narrative clinical documentation. However, note-based models are particularly vulnerable to temporal and lexical leakage, where documentation artifacts encode future clinical decisions and inflate apparent predictive performance. Such behavior poses substantial risks for real-world deployment, where overconfident or temporally invalid predictions can disrupt clinical workflows and compromise patient safety. This study focuses on system-level design choices required to build safe and deployable clinical NLP under temporal leakage constraints. We present a lightweight auditing pipeline that integrates interpretability into the model development process to identify and suppress leakage-prone signals prior to final training. Using next-day discharge prediction after elective spine surgery as a case study, we evaluate how auditing affects predictive behavior, calibration, and safety-relevant trade-offs. Results show that audited models exhibit more conservative and better-calibrated probability estimates, with reduced reliance on discharge-related lexical cues. These findings emphasize that deployment-ready clinical NLP systems should prioritize temporal validity, calibration, and behavioral robustness over optimistic performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。