提出几何度量方法,评估神经网络表示中类别的线性可分性。
A Geometric Measure of Linear Separability for Neural Representations

- 基于仿射半空间搜索,量化目标类在表示空间中的分离程度。
- 揭示了线性重参数化与信息丢失对可分性的不同影响。
- 适用于真实数据集,可诊断深度模型各组件的类别侵入问题。
现代神经分类器常依赖线性读出,但现有预测指标无法刻画此类读出所基于表示的类别几何特性。本文提出方向线性可分性度量(LSM),一种针对单侧仿射可分性的有限样本诊断工具。对于目标类别A和竞争类别集合B,LSM搜索包含所有A样本的仿射半空间,并衡量必须保留在目标侧的最小竞争样本侵入量,再归一化于|A|。该度量具有不对称性、类别特异性、目标归一化特性,适用于从神经网络提取的有限表示。我们建立了其支撑超平面表征,关联其与最优仿射分类准确率,并证明其在满秩线性嵌入下的不变性。这些结果可区分线性重参数化与信息损失或非线性几何变换带来的变化。此外,我们提出了基于惩罚项的仿射搜索方法,用于高维特征中类别级LSM的估计,报告值基于原始离散保留与违反准则计算。最后,我们分析了坐标门控非线性作为有限样本几何算子,并通过实证使用LSM诊断常见深度学习组件与架构中的类别级侵入现象。
原文摘要 · Abstract (English)
Modern neural classifiers commonly rely on linear readouts, yet predictive metrics alone do not characterize the class-wise geometry of the representations on which such readouts operate. We introduce the directional linear separability measure (LSM), a finite-sample diagnostic for one-sided affine separability. For a target class A and a competing set B, LSM searches over affine halfspaces that contain all samples in A and measures the smallest competing-sample intrusion that must remain on the target side, normalized by |A|. The resulting quantity is asymmetric, class-wise, target-normalized, and applicable to finite representations extracted from neural networks. We establish its supporting-hyperplane characterization, relate it to optimal affine classification accuracy, and prove invariance under full-rank linear embeddings. These results separate changes caused by linear reparameterization from those caused by information loss or nonlinear geometric transformations. We also give a penalty-based affine search for estimating class-wise LSM in high-dimensional features, with reported values computed from the original discrete preservation and violation criterion. Finally, we analyze coordinatewise gated nonlinearities as finite-sample geometric operators and empirically use LSM to diagnose class-wise intrusion across common deep-learning components and architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。