arXiv:2508.01219cs.CVcs.LG2025-08被引 1

用特征基重构网络权重,让模型更稳定、更快、更准确

Eigen Neural Network: Unlocking Generalizable Vision with Eigenbasis

  • 用共享的正交特征基重参数化权重,让学习过程更有序
  • 在ImageNet上超越现有方法,跨模态检索性能创纪录
  • 可无反向传播训练,速度提升2倍以上且精度更高

深度神经网络的成功依赖于梯度优化,但常因权重结构杂乱导致特征模糊、学习效率下降。为此,本文提出特征基神经网络(ENN),将每层权重重参数化为共享的、可学习的正交特征基。该设计从原理上强制权重动态解耦且对齐,无需依赖正则化。结合标准反向传播,ENN在大规模图像分类基准(如ImageNet)上持续领先,并在跨模态图文检索任务中设定新基准。此外,其结构支持高效无反向传播的局部学习变体ENN-ℓ,不仅突破反向传播的串行瓶颈,实现超过2倍训练加速,还意外超越端到端反向传播的精度。该工作揭示了一种直接修复反向传播表征缺陷的新架构范式,显著提升性能并实现更高效的并行训练。

原文摘要 · Abstract (English)

The remarkable success of Deep Neural Networks(DNN) is driven by gradient-based optimization, yet this process is often undermined by its tendency to produce disordered weight structures, which harms feature clarity and degrades learning dynamics. To address this fundamental representational flaw, we introduced the Eigen Neural Network (ENN), a novel architecture that reparameterizes each layer's weights in a layer-shared, learned orthonormal eigenbasis. This design enforces decorrelated, well-aligned weight dynamics axiomatically, rather than through regularization, leading to more structured and discriminative feature representations. When integrated with standard BP, ENN consistently outperforms state-of-the-art methods on large-scale image classification benchmarks, including ImageNet, and its superior representations generalize to set a new benchmark in cross-modal image-text retrieval. Furthermore, ENN's principled structure enables a highly efficient, backpropagation-free(BP-free) local learning variant, ENN-$\ell$. This variant not only resolves BP's procedural bottlenecks to achieve over 2$\times$ training speedup via parallelism, but also, remarkably, surpasses the accuracy of end-to-end backpropagation. ENN thus presents a new architectural paradigm that directly remedies the representational deficiencies of BP, leading to enhanced performance and enabling a more efficient, parallelizable training regime.

神经网络架构特征基训练加速无反向传播

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。