用分布匹配方法让自监督学习更稳定，提升特征质量与迁移能力。
Bringing Generative Learning to Representation Learning: Self-Supervised Transfer Learning as Distribution Matching
- 将表征学习建模为分布匹配，设计目标分布并用马洛斯距离衡量偏差。
- 在图像基准上实现流形校正、细粒度结构保留及跨标签空间迁移。
- 理论证明了分类误差与类别中心分离的关系，具非渐近保证。
多数自监督学习目标虽能防止表征坍缩，但未明确目标表征的具体形式。本文将表征学习定义为分布匹配（Distribution Matching, DM），学习一个对数据增强不变的编码器，使其诱导的分布与显式几何参考分布一致。该参考分布规定了理想表征应具备的形态，而独立选择的差异度量（此处采用马洛斯距离）则用于量化偏离程度。DM框架揭示了一种方向性逆关系：生成学习从可处理的参考分布映射到数据，而表征学习则从数据映射至预设的参考分布。我们建立了总体目标与类别中心分离及分类误差之间的联系，并给出了非渐近神经筛保证。仿真与图像基准测试表明，该方法能实现流形校正、细粒度结构保留以及跨标签空间的迁移性能提升。
原文摘要 · Abstract (English)
Most self-supervised learning objectives defend against collapse but leave the target representation law unspecified. We formulate representation learning as Distribution Matching (DM), learning an augmentation-invariant encoder whose induced law matches an explicit geometric reference. The reference law specifies what the learned representation distribution should look like, whereas a separately chosen discrepancy determines how deviations from this target are measured; here we use Mallows distance. The DM framework reveals a directional inverse: generative learning maps a tractable reference to data, whereas representation learning maps data to a designed reference law. We connect the population objective to class-centre separation and classification error and prove a non-asymptotic neural-sieve guarantee. Simulations and image benchmarks show manifold rectification, fine-grained structure and transfer across label spaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。