验证方式不同，预测模型的政策价值差异巨大。
Temporal Validation Changes the Apparent Public-Health Utility of Under-Five Mortality Prediction in Bangladesh: A Four-Round DHS Machine-Learning Study
- 用时间顺序验证替代随机验证，更贴近真实应用效果。
- 时间验证下模型敏感度42.8%，每7.6人筛查1名高危儿童。
- 建议报告敏感度、阳性预测值和筛查人数，指导政策制定。
背景：尽管孟加拉国在五岁以下儿童死亡率方面取得进展，但地区间仍不均衡。基于DHS的预测模型可用于指导针对性随访，但仅当验证方式反映未来实际应用时才有效。本研究分析了四个孟加拉国DHS调查轮次（2011–2022年；33,962名儿童；1,290例死亡），采用26个特征的流程和三种模型类别，在四种验证策略下评估性能，包括跨调查的时间验证（训练集为2011+2014，校准集为2017，测试集为2022）。通过遗传算法神经架构搜索选出32单元的ELU多层感知机。使用2,000次自举抽样计算AUROC；筛查效用以固定筛查容量下的敏感度、阳性预测值（PPV）和需筛查人数（NNS）衡量。结果：验证方式对公共卫生解释的影响大于模型类型。神经架构搜索的MLP AUROC在0.669（仅2022年随机验证）到0.775（合并随机验证）之间，时间验证为0.730。在2022年前10%的时间阈值下，识别出355例死亡中的152例（敏感度42.8%，PPV 13.2%，NNS 7.6）。不同验证设计的NNS范围为5.6至11.0。结论：验证方式的选择比模型架构对筛查工作量和政策价值的影响更大。时间验证可支持对随访与转诊需求的可靠估计；基于DHS的儿童死亡率研究在用于政策前应报告敏感度、PPV和NNS。
原文摘要 · Abstract (English)
Background: Under-five mortality in Bangladesh remains uneven despite national progress. DHS-based prediction models may guide targeted follow-up, but only if validation reflects future use. We examined how validation design changes apparent prediction performance. Methods: Four BDHS rounds (2011-2022; 33,962 children; 1,290 deaths) were analysed with a 26-feature pipeline and three model classes under four validation regimes, including cross-survey temporal validation (train 2011+2014, calibrate 2017, test 2022). A 32-unit ELU multilayer perceptron was selected via genetic-algorithm neural architecture search. AUROC used 2,000 bootstrap resamples; screening utility used sensitivity, PPV, and number needed to screen (NNS) at fixed capacity. Results: Validation regime altered public-health interpretation more than model class. NAS MLP AUROC ranged from 0.669 (2022-only random) to 0.775 (pooled random), with temporal AUROC 0.730. At the top-10% temporal threshold, NAS identified 152/355 deaths in 2022 (sensitivity 42.8%, PPV 13.2%, NNS 7.6). NNS across designs ranged from 5.6 to 11.0. Conclusions: Validation-regime choice changed screening workload and apparent policy value more than architecture. Temporal validation supports defensible estimates of follow-up and referral demand; DHS child-mortality studies should report sensitivity, PPV, and NNS before programmatic use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。