arXiv:2605.20302cs.LGcs.CV2026-05

通过球面原型学习,让分类模型精确逼近理论最优的神经坍缩。

Neural Collapse by Design: Learning Class Prototypes on the Hypersphere

论文配图:Neural Collapse by Design: Learning Class Prototypes on the Hypersphere
图 1 · 摘自论文原文
  • 在单位超球面上优化类原型,设计新损失函数实现统一优化。
  • 在四个基准上准确率超越交叉熵,95%以上逼近神经坍缩,迭代次数减少92.5%。
  • 消除线性探测冗余,提升迁移性能与抗干扰能力,适合高精度分类场景。

监督分类存在理论最优解——神经坍缩(NC),但现有两大主流方法均无法达到:交叉熵(CE)未约束径向自由度,导致退化几何;有监督对比学习(SCL)虽在预训练中逼近NC,但在后续线性探测阶段丢失该结构。本文揭示二者实为同一方法的不同表现,均在单位超球面上对比原型。从CE角度,提出NTCE和NONL两种归一化损失,引入对比学习缺失的负样本集规模大、对齐与均匀性解耦等要素;从SCL角度,证明其目标在训练全程即优化出以类均值嵌入为权重的合理分类器,故线性探测既冗余又有害。实验表明,在四个基准(含ImageNet-1K)上,NTCE与NONL超越CE准确率,接近95%以上的神经坍缩,仅用7.5%的迭代次数即可达成4/5指标,而固定原型的SCL可替代耗时的分类器训练。所学几何结构使迁移学习平均相对提升5.5%,严重类别不平衡下达8.7%,且在ImageNet-C上增强鲁棒性。本工作将监督学习重构为可设计地实现神经坍缩的原型学习。

原文摘要 · Abstract (English)

Supervised classification has a theoretical optimum, Neural Collapse (NC), yet neither of its two dominant paradigms reaches it in practice. Cross entropy (CE) leaves radial degrees of freedom unconstrained and converges to a degenerate geometry, while supervised contrastive learning (SCL) drives features toward NC during pretraining but discards this structure in a post hoc linear probing phase. We show that both paradigms are different appearances of the same method that contrasts prototypes on the unit hypersphere, and that closing the gap requires fixing each at its point of failure. From the CE side, we propose NTCE and NONL, two normalized losses that import contrastive optimization's missing ingredients into classifier learning: a large effective negative set and decoupled alignment and uniformity terms. From the SCL side, we prove that SCL's objective already optimizes throughout training for a principled classifier whose weights are the class mean embeddings, making linear probing both redundant and harmful. Empirically, on four benchmarks including ImageNet-1K, NTCE and NONL surpass CE accuracy, closely approximate NC ($\geq 95\%$), and match CE's converged NC on 4/5 metrics in under $7.5\%$ of its iterations, while SCL with fixed prototypes matches linear probing without the hours-long classifier training phase. The learned geometry yields $+5.5\%$ mean relative improvement in transfer learning, up to $+8.7\%$ under severe class imbalance, and improved robustness to corruptions on ImageNet-C. Our work recasts supervised learning as prototype learning on the hypersphere, with NC reached by design.

神经坍缩原型学习分类器优化超球面

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。