arXiv:2608.30040stat.MLcs.LG2026-08

统一建模缺失数据、测量误差和异质性,提升复杂场景下数据质量。

A Deep Latent Variable Framework for Jointly Modeling Missingness, Measurement Error, and Heterogeneity

  • 用深度潜在变量框架联合处理缺失、误差与群体差异
  • 在多种缺失机制下表现优于现有方法,提升估计效率
  • 适合高维医疗等噪声与不完整数据场景

缺失数据、测量误差和人群异质性是现代观察性研究与机器学习应用中的普遍挑战。这些问题是共存且相互影响的,但现有方法常将其分开处理。本文提出一种统一的概率框架,利用深度潜在变量表示联合建模这些问题。方法结合新型分层树路由变分自编码器、模式感知潜在表征和基于校准的去噪机制,可支持MCAR、MAR和MNAR等多种缺失机制,同时学习子群特异性和全局共享的潜在结构。引入的重构路由机制使相关子群可选择性共享参数,兼具灵活性与统计效率。模拟实验显示,在复杂的异质性缺失与测量误差设置下,性能显著优于现有深度生成插补方法。该框架为现代医疗等高维场景下的噪声与不完整数据学习提供了严谨方法。

原文摘要 · Abstract (English)

Missing data, measurement error, and population heterogeneity are pervasive challenges in analyzing data arising from modern observational studies and machine learning applications. Although these problems frequently coexist and interact, they are often treated separately in existing works. We propose a unified probabilistic framework that jointly addresses these issues utilizing deep latent variable representation. The proposed method integrates a novel hierarchical tree-routed variational autoencoder with pattern-aware latent representations and calibration-based denoising. The framework accommodates missing data mechanisms, including MCAR, MAR, and MNAR, while simultaneously learning subgroup-specific and globally shared latent structure. The introduced reconvergent routing mechanism enables selective parameters to be shared across related subpopulations, which offers flexibility as well as improved statistical efficiency. Simulation studies demonstrate substantial improvements over existing deep generative imputation approaches under complex heterogeneous missingness and measurement-error settings. The proposed framework provides a principled approach for learning from noisy and incomplete data in modern healthcare and other high-dimensional applications.

潜在变量缺失数据异质性深度学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。