用强化学习自动优化生存分析数据预处理,提升模型效果。
CleanSurvival: Automated data preprocessing for time-to-event models using reinforcement learning
- 基于Q-learning构建预处理决策框架,自动选择最佳数据清洗组合。
- 在真实数据集上相比基线方法显著提升生存模型预测性能。
- 适合需要高效、可靠生存分析的医疗研究者与数据科学家。
数据预处理在机器学习中常被忽视,但其对模型性能影响巨大。尽管自动化机器学习(AutoML)已将预处理集成到分类与回归任务中,但在处理右删失数据的生存分析任务中仍缺乏专门的自动化方案。为此,本文提出CleanSurvival,一种基于强化学习的生存分析专用预处理框架。该框架扩展自Learn2Clean的Q-learning方法,可自动选择最优的数据插补、异常值检测与特征提取组合,适用于Cox模型、随机森林、神经网络等时间-事件模型。在真实数据集上的实验表明,该方法相比简单基线显著提升预测性能,运行时行为具有条件依赖性,最佳覆盖的基准单元最易解释。模拟研究进一步验证了其在不同缺失率和噪声水平下的有效性。随着机器学习应用日益广泛,将AutoML扩展至生存分析等复杂模型至关重要。CleanSurvival等工具可加速生存研究,提升结果稳健性。
原文摘要 · Abstract (English)
Data preprocessing is often paid little attention in machine learning, despite its potentially significant impact on model performance. While automated machine learning pipelines are starting to recognise and integrate data preprocessing into their solutions for classification and regression tasks, this integration is lacking for more specialised tasks like time-to-event models for censored data. As a result, survival analysis not only faces the general challenges of data preprocessing but also suffers from the lack of tailored, automated solutions in this area. To address this gap, this paper presents CleanSurvival, a reinforcement-learning-based solution for optimizing preprocessing pipelines, extended specifically for survival analysis. The framework can handle continuous and categorical variables. It builds upon Learn2Clean's Q-learning to select which combination of data imputation, outlier detection and feature extraction techniques achieves optimal performance for a Cox, random forest, neural network or user-supplied time-to-event model. Experimental benchmarks on real-world datasets show that the Q-learning-based data preprocessing can improve predictive performance relative to simple baselines, while runtime behaviour is condition-dependent and most clearly interpretable in the best-covered benchmark cells. Furthermore, a simulation study demonstrates effectiveness across different types and levels of missingness and noise. With an increase in the use of machine learning, it becomes important to generalise AutoML pipelines to a variety of models now present, including survival analysis. Tools like CleanSurvival, which integrate preprocessing for survival analysis, can make survival studies faster and easier to perform, while also yielding more robust results.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。