用电子病历预测血压，发现模型在不同医院间泛化失败。
When Validation Fails: Cross-Institutional Blood Pressure Prediction and the Limits of Electronic Health Record-Based Models
- 构建集成框架，严格防止数据泄露，从MIMIC-III数据集学习血压预测。
- 外部验证在eICU数据集上表现大幅下滑，收缩压R²从0.248降至-0.024。
- 揭示跨机构障碍：测量方式、患者群体和特征分布差异是核心瓶颈。
尽管外部验证对评估医疗机器学习模型的真实可行性至关重要，但其应用仍极为罕见。我们开发了一个集成框架，基于电子健康记录预测血压，并实施了严格的防数据泄露机制。在MIMIC-III数据集上的内部验证显示，收缩压（R² = 0.248，RMSE = 14.84 mmHg）和舒张压（R² = 0.297，RMSE = 8.27 mmHg）表现中等。然而，在eICU数据集上的外部验证揭示了显著的泛化挑战：基线收缩压性能从R² = 0.248下降至-0.024，RMSE从14.84升至18.69 mmHg。为排除特征插补的干扰，我们仅使用16个普遍可用特征进行交集实验，结果更差（R² = -0.115，RMSE = 17.32 mmHg），证明插补并非主因。尝试后处理校正（线性与保序重标定，R²范围-0.170至0.024）及通过协变量偏移重加权实现领域自适应（R² = -0.141），均未显著改善。根因分析识别出三大泛化障碍：（1）即使标准生理变量也存在机构特异性特征分布；（2）患者人群差异导致独特病理生理特征；（3）测量协议差异造成不可转移的学习模式。研究表明，强内部性能无法保证跨机构部署成功。透明报告验证失败对于设定预测模型现实预期至关重要。代码已公开于https://github.com/mdbasit897/ehr-bp-ensemble。
原文摘要 · Abstract (English)
External validation remains rare in healthcare machine learning despite being critical for establishing real-world feasibility. We developed an ensemble framework to predict blood pressure from electronic health records, incorporating rigorous data leakage prevention. Internal validation on the MIMIC-III dataset yielded moderate performance for systolic (R^2 = 0.248, RMSE = 14.84 mmHg) and diastolic (R^2 = 0.297, RMSE = 8.27 mmHg) blood pressure. However, external validation on the eICU dataset revealed substantial generalization challenges. Baseline systolic performance dropped significantly from R^2 = 0.248 to -0.024, with RMSE increasing from 14.84 to 18.69 mmHg. To address potential confounding from feature imputation, we conducted an intersection-only experiment using 16 universally available features; this yielded worse external performance (R^2 = -0.115, RMSE = 17.32 mmHg), proving imputation artifacts were not the primary cause. Attempts at post-hoc correction, including linear and isotonic recalibration (R^2 ranging from -0.170 to 0.024) and domain adaptation via covariate shift reweighting (R^2 = -0.141), showed limited gains. This highlights fundamental cross-institutional barriers. Our root-cause analysis identified three primary obstacles to generalizability: (1) site-specific feature distributions, even among standard physiological variables; (2) underlying patient population differences with unique pathophysiologies; and (3) institutional variations in measurement protocols creating non-transferable learned patterns. These findings demonstrate that strong internal performance cannot guarantee cross-institutional deployment success. Transparent reporting of validation failures is essential for setting realistic expectations for predictive models. Code is available at https://github.com/mdbasit897/ehr-bp-ensemble.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。