arXiv:2608.14638cs.LGmath.PR2026-08

研究自编码器的固定点与混沌边缘,揭示其稳定性机制。

Randomly initialized autoencoders: fixed points and edge-of-chaos

  • 用随机矩阵理论分析自编码器固定点稳定性
  • 提出局部与全局混沌边缘控制输入扰动
  • 为初始化提供理论依据,适合深度学习研究者

本文研究自编码器这一特殊类型的深度神经网络(DNN),其性能可通过固定点来刻画。该视角自然引出固定点的存在性、稳定性及吸引域的问题。这些问题通过自编码器的压缩性质得到解答,并与混沌边缘(EoC)概念密切相关。混沌边缘是描述随机初始化网络中信号传播有序与混沌临界状态的重要理论概念。在该临界区域初始化可带来理论与实践上的优势,如对输入扰动的鲁棒性。以往的混沌边缘理论多基于均场平均方法适用于广泛神经网络。本文针对自编码器改进该概念,提出局部与全局混沌边缘,分别控制小规模和任意规模的输入扰动。自编码器的稳定性分析属于随机矩阵理论(RMT)中的非线性问题。局部混沌边缘采用RMT谱技术分析,全局混沌边缘则借助高斯过程的Sudakov-Fernique不等式进行研究。

原文摘要 · Abstract (English)

In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises questions of existence, stability, and basins of attraction of these fixed points. These questions are addressed via the contractive properties of autoencoders, and are closely related to the notion of edge-of-chaos. Edge-of-chaos (EoC) is an important notion in the theory of DNNs. It describes the critical regime separating ordered and chaotic signal propagation through a randomly initialized network. Initialization at or near this critical regime offers several theoretical and practical advantages, including stability of the network w.r.t. perturbations of the input. EoC was previously introduced for broad classes of neural networks using mean-field averaging methods. In this paper we modify the notion of EoC for the study of autoencoders. Specifically, we introduce local and global EoC for autoencoders that control local (small) and global (arbitrary) perturbations of the input respectively. The study of stability of autoencoders falls within the scope of nonlinear problems in Random Matrix Theory (RMT). Our analysis of local EoC is based on spectral techniques of RMT, whereas global EoC is studied by employing Sudakov-Fernique inequality for Gaussian processes.

自编码器混沌边缘随机矩阵稳定性分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。