用核方法改进自监督学习的表征稳定性，提升模型泛化能力。
KerJEPA: Kernel Discrepancies for Euclidean Self-Supervised Learning
- 采用核正则化机制替代传统欧氏空间正则，增强表征学习灵活性。
- 在高维极限下实现 sliced MMD 的闭式解，提升训练稳定性。
- 适用于需要强泛化能力的自监督视觉任务，如图像分类与迁移学习。
自监督联合嵌入预测架构（JEPAs）的最新进展表明，将欧氏表示正则化至各向同性高斯先验可显著提升训练稳定性和下游泛化性能。本文提出一类新型灵活的 KerJEPA 方法，即基于核函数的自监督学习算法。其中一种实例对应于近期提出的 LeJEPA Epps-Pulley 正则化器，其通过高斯先验与高斯核近似切片最大均值差异（sliced MMD）。通过扩展可行核函数与先验分布的类别，并推导切片 MMD 在高维下的闭式极限，我们构建了具备更优训练稳定性与设计灵活性的替代 KerJEPAs。
原文摘要 · Abstract (English)
Recent breakthroughs in self-supervised Joint-Embedding Predictive Architectures (JEPAs) have established that regularizing Euclidean representations toward isotropic Gaussian priors yields provable gains in training stability and downstream generalization. We introduce a new, flexible family of KerJEPAs, self-supervised learning algorithms with kernel-based regularizers. One instance of this family corresponds to the recently-introduced LeJEPA Epps-Pulley regularizer which approximates a sliced maximum mean discrepancy (MMD) with a Gaussian prior and Gaussian kernel. By expanding the class of viable kernels and priors and computing the closed-form high-dimensional limit of sliced MMDs, we develop alternative KerJEPAs with a number of favorable properties including improved training stability and design flexibility.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。