提出新损失函数，让深度线性判别分析兼具判别性能与概率可解释性。
Deep Linear Discriminant Analysis Revisited
- 引入DNLL损失，通过惩罚混合密度抑制多类共存区域。
- 训练后特征空间清晰分离，在合成数据和图像基准上准确率媲美softmax。
- 恢复了模型的概率解释性，预测置信度更可信，适合需要可靠概率的场景。
我们发现，对于无约束的深度线性判别分析（LDA）分类器，最大似然训练会产生病态解：类别均值趋于聚集，协方差坍缩，学习到的表示几乎不具备判别性。相反，交叉熵训练虽能获得高精度，但使分类头与底层生成模型脱钩，导致参数估计严重不一致。为调和生成结构与判别性能，我们提出判别负对数似然（DNLL）损失，其在LDA似然基础上增加对混合密度的简单惩罚。DNLL可理解为标准LDA负对数似然加上一个显式抑制多个类别同时高概率出现区域的项。使用DNLL训练的深度LDA能生成清晰、良好分离的隐空间，在合成数据和标准图像基准上达到与softmax分类器相当的测试准确率，并显著提升预测概率校准效果，重新赋予深度判别模型一致的概率解释。
原文摘要 · Abstract (English)
We show that for unconstrained Deep Linear Discriminant Analysis (LDA) classifiers, maximum-likelihood training admits pathological solutions in which class means drift together, covariances collapse, and the learned representation becomes almost non-discriminative. Conversely, cross-entropy training yields excellent accuracy but decouples the head from the underlying generative model, leading to highly inconsistent parameter estimates. To reconcile generative structure with discriminative performance, we introduce the \emph{Discriminative Negative Log-Likelihood} (DNLL) loss, which augments the LDA log-likelihood with a simple penalty on the mixture density. DNLL can be interpreted as standard LDA NLL plus a term that explicitly discourages regions where several classes are simultaneously likely. Deep LDA trained with DNLL produces clean, well-separated latent spaces, matches the test accuracy of softmax classifiers on synthetic data and standard image benchmarks, and yields substantially better calibrated predictive probabilities, restoring a coherent probabilistic interpretation to deep discriminant models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。