将自监督学习统一为潜在分布匹配，解释了现有方法并指导新模型设计。
Understanding Self-Supervised Learning via Latent Distribution Matching
- 把自监督学习看作潜在分布匹配，兼顾对齐与均匀性。
- 推导出无需采样的非线性贝叶斯滤波模型，适用于高维时间序列。
- 证明预测型方法在弱假设下可获得可识别的潜在表示，适合研究者参考。
自监督学习（SSL）能在复杂数据中提取通用潜在表示，但缺乏统一的理论框架来解释现有方法并指导新方法的设计。本文将SSL视为潜在分布匹配（LDM）：在假设的潜在模型下最大化表示的对数概率（对齐），同时最大化潜在熵以防止坍缩（均匀性）。这一视角统一了独立成分分析与对比、非对比及预测型自监督学习方法，包括停止梯度策略。基于LDM，我们推导出一种非线性、无需采样的贝叶斯滤波模型，采用卡尔曼式预测器处理高维时间序列。进一步证明，在弱假设下，预测型LDM能获得可识别的潜在表示，即使使用非线性预测器。总体而言，LDM澄清了现有方法背后的假设，并为新方法的设计提供了原则性指导。
原文摘要 · Abstract (English)
Self-supervised learning (SSL) excels at finding general-purpose latent representations from complex data, yet lacks a unifying theoretical framework that explains the diverse existing methods and guides the design of new ones. We cast SSL as latent distribution matching (LDM): learning representations that maximize their log-probability under an assumed latent model (alignment), while maximizing latent entropy to prevent collapse (uniformity). This view unifies independent component analysis with contrastive, non-contrastive, and predictive SSL methods, including stop gradient approaches. Leveraging LDM, we derive a nonlinear, sampling-free Bayesian filtering model with a Kalman-based predictor for high-dimensional timeseries. We further prove that predictive LDM yields identifiable latent representations under mild assumptions, even with nonlinear predictors. Overall, LDM clarifies the assumptions behind established SSL methods and provides principled guidance for developing new approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。