用混合型数据统一编码+Mamba网络,高效修复高维缺失数据。
RefiDiff: Progressive Refinement Diffusion for Efficient Missing Data Imputation
- 先预测后生成,分步精修提升修复精度
- 在九个真实数据集上优于现有方法,尤其擅长处理非随机缺失
- 无需调参即可稳定运行,适合复杂缺失场景
高维混合类型数据中的缺失值在非随机缺失(MNAR)机制下难以修复,现有方法难以融合局部与全局特征。本文提出RefiDiff框架,结合局部机器学习预测与新型基于Mamba的去噪网络,以低计算成本捕捉特征与样本间的长程依赖。该方法融合预测与生成范式:通过预精修提供初始填充,再经后精修优化结果,提升稳定性和准确性。通过将混合类型数据统一编码为标记,实现无需架构或超参数调整的鲁棒修复。在九个真实世界数据集上的广泛评估表明,RefiDiff在各类缺失设定下均优于当前最先进方法,在MNAR场景和泛化能力上表现突出,具备强鲁棒性、可扩展性及对复杂缺失模式的有效处理能力。
原文摘要 · Abstract (English)
Missing values in high-dimensional, mixed-type datasets pose significant challenges for data imputation, particularly under Missing Not At Random (MNAR) mechanisms. Existing methods struggle to integrate local and global data characteristics, limiting performance in MNAR and high-dimensional settings. We propose an innovative framework, RefiDiff, combining local machine learning predictions with a novel Mamba-based denoising network efficiently capturing long-range dependencies among features and samples with low computational complexity. RefiDiff bridges the predictive and generative paradigms of imputation, leveraging pre-refinement for initial warm-up imputations and post-refinement to polish results, enhancing stability and accuracy. By encoding mixed-type data into unified tokens, RefiDiff enables robust imputation without architectural or hyperparameter tuning. RefiDiff outperforms state-of-the-art (SOTA) methods across missing-value settings, demonstrating strong performance in MNAR settings and superior out-of-sample generalization. Extensive evaluations on nine real-world datasets demonstrate its robustness, scalability, and effectiveness in handling complex missingness patterns.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。