arXiv:2511.12278stat.MLcs.LG2025-11NeurIPS被引 4

通过强制特征均匀性,提升对比学习在噪声中的信号恢复能力

PCA++: How Uniformity Induces Robustness to Background Noise in Contrastive Learning

  • 引入硬均匀性约束的对比PCA,通过投影特征协方差为单位阵来抑制背景噪声
  • 在高维和强噪声下仍稳定,能准确恢复条件不变的共享信号子空间
  • 适合处理带结构噪声的高维数据,如单细胞转录组和噪声图像数据

高维数据常被结构化背景噪声掩盖低维信号,标准PCA效果受限。受对比学习启发,我们研究从正样本对中恢复共享信号子空间的问题,这些样本具有相同信号但背景不同。基准方法PCA+仅使用对齐的对比学习,在背景变化温和时有效,但在强噪声或高维情形下失效。为此,我们提出PCA++,一种施加硬均匀性约束的对比PCA,强制投影特征具有单位协方差。PCA++可通过广义特征值问题求得闭式解,在高维下保持稳定,并可证明地正则化以抵抗背景干扰。我们在固定比率和增长尖峰两种高维渐近场景下给出精确分析,揭示均匀性在鲁棒信号恢复中的作用。实验表明,PCA++在模拟数据、带噪MNIST和单细胞转录组数据上均优于标准PCA和仅对齐的PCA+,可靠恢复条件不变结构。更广泛地,本工作阐明了均匀性在对比学习中的作用:显式特征分散能抵御结构化噪声,增强鲁棒性。

原文摘要 · Abstract (English)

High-dimensional data often contain low-dimensional signals obscured by structured background noise, which limits the effectiveness of standard PCA. Motivated by contrastive learning, we address the problem of recovering shared signal subspaces from positive pairs, paired observations sharing the same signal but differing in background. Our baseline, PCA+, uses alignment-only contrastive learning and succeeds when background variation is mild, but fails under strong noise or high-dimensional regimes. To address this, we introduce PCA++, a hard uniformity-constrained contrastive PCA that enforces identity covariance on projected features. PCA++ has a closed-form solution via a generalized eigenproblem, remains stable in high dimensions, and provably regularizes against background interference. We provide exact high-dimensional asymptotics in both fixed-aspect-ratio and growing-spike regimes, showing uniformity's role in robust signal recovery. Empirically, PCA++ outperforms standard PCA and alignment-only PCA+ on simulations, corrupted-MNIST, and single-cell transcriptomics, reliably recovering condition-invariant structure. More broadly, we clarify uniformity's role in contrastive learning, showing that explicit feature dispersion defends against structured noise and enhances robustness.

对比学习降维噪声鲁棒高维统计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。