用量子启发方法改进数据填补,让缺失值更真实可信。
Quantum-Inspired Optimization Process for Data Imputation
- 结合主成分分析与量子旋转,用经典优化器调整填补结果。
- 填补后数据分布与原数据差距缩小超85%,统计检验显著改善。
- 适合医疗等对数据质量要求高的场景,避免虚假集中问题。
数据填补是数据预处理中的关键步骤,尤其针对存在缺失或不可靠值的数据库。本研究提出一种新型量子启发填补框架,在包含生物上不合理缺失值的UCI糖尿病数据集上进行评估。该方法将主成分分析(PCA)与量子辅助旋转结合,通过无梯度经典优化器(COBYLA、模拟退火、差分进化)优化,重建缺失值并保持统计保真性。重建值被约束在原始特征分布±2个标准差范围内,避免值过度聚集于均值附近。相比均值法、KNN和MICE等传统方法(p值>0.99),本方法实现超过85%的Wasserstein距离降低,以及0.18至0.22之间的柯尔莫哥洛夫-斯米尔诺夫检验p值,具有显著统计提升。该方法消除零值伪影,增强填补数据的真实性和变异性。通过将量子启发变换与可扩展的经典框架结合,为医疗及人工智能流程中数据质量与完整性至关重要的领域提供了稳健的填补解决方案。
原文摘要 · Abstract (English)
Data imputation is a critical step in data pre-processing, particularly for datasets with missing or unreliable values. This study introduces a novel quantum-inspired imputation framework evaluated on the UCI Diabetes dataset, which contains biologically implausible missing values across several clinical features. The method integrates Principal Component Analysis (PCA) with quantum-assisted rotations, optimized through gradient-free classical optimizers -COBYLA, Simulated Annealing, and Differential Evolution to reconstruct missing values while preserving statistical fidelity. Reconstructed values are constrained within +/-2 standard deviations of original feature distributions, avoiding unrealistic clustering around central tendencies. This approach achieves a substantial and statistically significant improvement, including an average reduction of over 85% in Wasserstein distance and Kolmogorov-Smirnov test p-values between 0.18 and 0.22, compared to p-values > 0.99 in classical methods such as Mean, KNN, and MICE. The method also eliminates zero-value artifacts and enhances the realism and variability of imputed data. By combining quantum-inspired transformations with a scalable classical framework, this methodology provides a robust solution for imputation tasks in domains such as healthcare and AI pipelines, where data quality and integrity are crucial.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。