揭示生存分析模型过参数化下的双下降现象及其机制
Understanding Overparametrization in Survival Models through Interpolation
- 定义插值与有限范数插值，解析模型过参数化条件
- 四类生存模型中仅部分实现插值，且泛化性能不随容量单调提升
- 为生存建模提供理论依据，适合关注模型泛化的研究者
经典统计学习理论预测测试损失与模型容量呈U型关系，源于偏差-方差权衡。现代机器学习揭示更复杂的双下降现象：在插值阈值附近损失达峰值后，继续增加模型容量时损失再次下降。尽管该现象在回归和分类任务中已有广泛研究，但在生存分析领域仍未知。本研究考察了四种代表性生存模型：DeepSurv、PC-Hazard、Nnet-Survival 和 N-MTLR。我们严格定义了基于损失的模型中的插值与有限范数插值两个关键特征，以理解双下降行为。结果表明，这四类模型中存在（或不存在）（有限范数）插值。研究揭示了似然函数形式与模型实现方式共同决定插值可行性，并指出过参数化不应被视为对生存模型无害。所有理论结论均通过数值实验验证，凸显了生存模型独特的泛化行为。
原文摘要 · Abstract (English)
Classical statistical learning theory predicts a U-shaped relationship between test loss and model capacity, driven by the bias-variance trade-off. Recent advances in modern machine learning have revealed a more complex pattern, double-descent, in which test loss, after peaking near the interpolation threshold, decreases again as model capacity continues to grow. While this behavior has been extensively analyzed in regression and classification, its manifestation in survival analysis remains unexplored. This study investigates overparametrization in four representative survival models: DeepSurv, PC-Hazard, Nnet-Survival, and N-MTLR. We rigorously define interpolation and finite-norm interpolation, two key characteristics of loss-based models to understand double-descent. We then show the existence (or absence) of (finite-norm) interpolation of all four models. Our findings clarify how likelihood-based losses and model implementation jointly determine the feasibility of interpolation and show that overparametrization should not be regarded as benign for survival models. All theoretical results are supported by numerical experiments that highlight the distinct generalization behaviors of survival models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。