通过径向高斯化提升自监督表示学习的多样性与信息量
Radial-VCReg: More Informative Representation Learning Through Radial Gaussianization
- 在VCReg基础上引入径向高斯化损失，使特征模长对齐卡方分布
- 在合成与真实数据集上显著降低高阶依赖，提升表示多样性
- 适合追求更高质量自监督表征的研究者或系统开发者
自监督学习旨在学习最具信息量的表示，但高维空间中的信息最大化受到维度灾难的制约。现有方法如VCReg通过正则化一阶和二阶特征统计量来缓解此问题，但无法实现最大熵。本文提出Radial-VCReg，通过引入径向高斯化损失，使特征模长与卡方分布对齐——这是高维高斯分布的本质特性。我们证明，相较于VCReg，Radial-VCReg能将更广泛的分布转化为正态分布。在合成数据与真实数据集上的实验表明，该方法持续提升了性能，有效降低了高阶依赖，促进了更具多样性和信息量的表示学习。
原文摘要 · Abstract (English)
Self-supervised learning aims to learn maximally informative representations, but explicit information maximization is hindered by the curse of dimensionality. Existing methods like VCReg address this by regularizing first and second-order feature statistics, which cannot fully achieve maximum entropy. We propose Radial-VCReg, which augments VCReg with a radial Gaussianization loss that aligns feature norms with the Chi distribution-a defining property of high-dimensional Gaussians. We prove that Radial-VCReg transforms a broader class of distributions towards normality compared to VCReg and show on synthetic and real-world datasets that it consistently improves performance by reducing higher-order dependencies and promoting more diverse and informative representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。