arXiv:2605.26900cs.LG2026-05被引 3

提出球面均匀嵌入,解决自监督学习中表征几何的理论缺陷。

SPHERE-JEPA: Spherical Prediction with Homogeneous Embeddings

论文配图:SPHERE-JEPA: Spherical Prediction with Homogeneous Embeddings
图 1 · 摘自论文原文
  • 用球面均匀性替代高斯先验,改进表征学习几何结构。
  • 在图像数据集上提升纹理检索准确率超6%,线性探测增益1.8%。
  • 适合关注表征几何与自监督学习理论的科研人员。

自监督学习中表征最优几何仍是一个基础性问题。近期研究发现,在欧氏空间中各向同性高斯嵌入可最小化下游预测风险。然而,对低维流形(如超球面)上分布的最优解尚未探索。本文将极小极大分析拓展至黎曼流形,揭示:在最坏情况假设下,k近邻与核岭回归均诱导超球面均匀性。具体而言,流形上的均匀分布是k近邻的最优解,而球面上的均匀分布对指数点积核与线性核的核岭回归最优。该理论表明高斯嵌入因密度非均匀导致近邻偏差,严重误导估计。为此,我们提出SPHERE-JEPA,基于LeJEPA的Cramér-Wold投影机制,强制实现球面均匀性而非高斯先验。实验表明,SPHERE-JEPA显著提升性能,纹理检索mAP提升超过6%,并在标准基准上持续优于或匹配LeJEPA,包括ImageNet-1K(ViT-B/14)线性探测增益+1.8%。

原文摘要 · Abstract (English)

A fundamental open question in self-supervised learning (SSL) is the explicit characterization of the optimal geometry of the learned representations. Recently, LeJEPA identified isotropic Gaussian embeddings as optimal for minimizing downstream prediction risk in Euclidean spaces. However, the corresponding problem for distributions supported on lower-dimensional manifolds, such as the hypersphere, remains unexplored. In this work, we demonstrate that extending this minimax analysis to smooth distributions on Riemannian manifolds fundamentally changes the optimal solution. We show that, under a worst-case formulation, both k-nearest neighbors and kernel ridge regression induce hyperspherical uniformity. More precisely, we show that uniform distributions on manifolds are optimal for k-nearest neighbors, and that the uniform distribution on the sphere is optimal for kernel ridge regression with both the exponential dot-product kernel and the linear kernel. This theoretical insight reveals a fundamental limitation of Gaussian embeddings: their non-uniform density induces anisotropic k-NN neighborhoods, severely biasing the estimator. To correct this, we introduce SPHERE-JEPA, a theoretically grounded SSL framework. We adapt LeJEPA's Cram{é}r-Wold projection mechanism to enforce hyperspherical uniformity rather than a Gaussian prior. Empirically, SPHERE-JEPA yields significant improvements, boosting texture retrieval mAP by over 6%, while consistently matching or outperforming LeJEPA on standard benchmarks-including a +1.8% linear probing gain on ImageNet-1K (ViT-B/14).

自监督学习表征几何球面嵌入

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。