arXiv:2608.20668cs.LGcs.AI2026-08

用球面流形学习让ReduNet层数减少10倍,参数量降为1/29。

Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning

论文配图:Lightweight Adaptive ReduNet via Hyperspherical Manifold Learning
图 1 · 摘自论文原文
  • 引入球面流形学习与自适应步长优化层间更新规则。
  • 在相同精度下,所需层数减少至原版的1/29,参数存储量大幅降低。
  • 适合对模型轻量化和可解释性有要求的研究者。

近年来提出的白盒神经网络ReduNet基于最大编码率缩减(MCR²)原理,通过逐层前向构建将原始数据转化为低维判别特征。与依赖反向传播的传统深度网络不同,ReduNet显式地从前一层特征推导每层参数,具有数学可解释性。然而,其逐层构建过程通常需要大量层数才能使MCR²目标稳定,导致展开模块参数存储量大。为此,本文提出LA-ReduNet,一种轻量级自适应架构,通过改进逐层更新规则,显著减少达到稳定状态所需的展开层数。具体而言,LA-ReduNet结合球面流形学习与自适应步长策略,使MCR²目标达到稳定所需的层数减少一个数量级。仿真结果表明,在保持相近分类精度的前提下,LA-ReduNet仅需约1/29的参数存储量即可使MCR²目标稳定。

原文摘要 · Abstract (English)

In recent years, a white-box neural network called ReduNet has been proposed, which employs the maximal coding rate reduction (MCR$^2$) principle to transform raw data into low-dimensional discriminative features via a forward layer-wise construction process. Unlike traditional deep networks that rely on backpropagation, ReduNet explicitly derives the parameters of each layer from the features of its preceding layer, offering a mathematically interpretable paradigm. However, this layer-wise construction often requires a large number of layers for the MCR$^2$ objective to reach a stable value, which increases the parameter storage of the unfolded module. To address this issue, we propose LA-ReduNet, a lightweight adaptive architecture that refines the layer-wise update rule and enables discriminative feature representations to be obtained with substantially fewer unfolded layers. Specifically, LA-ReduNet employs hyperspherical manifold learning and adaptive step sizes, thereby reducing by an order of magnitude the number of layers required for the MCR$^2$ objective to reach a stable value. Simulation results demonstrate that, while maintaining comparable classification accuracy, LA-ReduNet requires significantly fewer layers for the MCR$^2$ objective to reach a stable value. Remarkably, under the considered experimental settings, LA-ReduNet requires only approximately $1/29$ of the parameter storage of the unfolded ReduNet module for the MCR$^2$ objective to reach a stable value.

轻量化流形学习可解释性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。