通过贝叶斯动态调整子空间扩展,提升ReduNet的分类分离性。
ESS-ReduNet: Enhancing Subspace Separability of ReduNet via Dynamic Expansion with Bayesian Inference
- 基于贝叶斯推断动态控制子空间扩展,增强类别间分离度。
- 在ESR数据集上,SVM分类准确率提升47%,收敛速度提高10倍以上。
- 适合需要快速收敛和高可分特征表示的分类任务使用。
ReduNet是一种基于最大编码速率压缩原理的深度神经网络,将原始数据映射为低维线性可分特征表示。与传统深度学习不同,其参数逐层显式构建,每层参数由前一层特征推导而来。该模型不直接使用标签,而是利用类别子空间与样本间的相似性进行特征更新,可能导致更新方向错误,影响参数正确构建并降低收敛速度。为此,本文提出ESS-ReduNet,基于网络参数的几何解释,通过动态控制样本整体张成空间的扩展,增强各类别子空间的可分性;同时引入标签知识,结合贝叶斯推断促进子空间解耦;以条件数作为稳定性辅助判据,用于停止训练。在ESR、HAR、Covertype和Gas数据集上的实验表明,相比ReduNet,ESS-ReduNet收敛速度提升超过10倍。尤其在ESR数据集上,其特征经SVM分类准确率提升47%。
原文摘要 · Abstract (English)
ReduNet is a deep neural network model that leverages the principle of maximal coding rate \textbf{redu}ction to transform original data samples into a low-dimensional, linear discriminative feature representation. Unlike traditional deep learning frameworks, ReduNet constructs its parameters explicitly layer by layer, with each layer's parameters derived based on the features transformed from the preceding layer. Rather than directly using labels, ReduNet uses the similarity between each category's spanned subspace and the data samples for feature updates at each layer. This may lead to features being updated in the wrong direction, impairing the correct construction of network parameters and reducing the network's convergence speed. To address this issue, based on the geometric interpretation of the network parameters, this paper presents ESS-ReduNet to enhance the separability of each category's subspace by dynamically controlling the expansion of the overall spanned space of the samples. Meanwhile, label knowledge is incorporated with Bayesian inference to encourage the decoupling of subspaces. Finally, stability, as assessed by the condition number, serves as an auxiliary criterion for halting training. Experiments on the ESR, HAR, Covertype, and Gas datasets demonstrate that ESS-ReduNet achieves more than 10x improvement in convergence compared to ReduNet. Notably, on the ESR dataset, the features transformed by ESS-ReduNet achieve a 47\% improvement in SVM classification accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。