arXiv:2601.07257cs.LGcs.IT2026-01

揭示物理系统中隐藏的创新容量,解释为何传统能力指标偏低。

Innovation Capacity of Dynamical Learning Systems

  • 提出创新容量概念,分解可预测与非可预测信息分量
  • 证明容量守恒:总容量等于可观测协方差秩,且两部分互补
  • 大创新容量带来海量可区分历史,支持生成建模

在噪声物理储层中,经典信息处理容量 $C_{ ext{ip}}$ 衡量线性读出利用输入历史完成任务的能力,但其值常远小于读出协方差的实际秩。本文引入创新容量 $C_{ ext{i}}$,定义为正交于输入滤波的读出分量(即 Doob 创新,包含输入-噪声混杂)所分配的总容量。通过无基底的希尔伯特空间方法,证明容量守恒律 $C_{ ext{ip}} + C_{ ext{i}} = ext{rank}(oldsymbol{ ext{Σ}}_{XX}) \\[ \le d$,表明可预测与创新容量精确划分了可观测读出维度协方差 $oldsymbol{ ext{Σ}}_{XX} \in \mathbb{R}^{d \times d}$ 的秩。在线性高斯约翰逊-奈奎斯特情形下,$\boldsymbol{ ext{Σ}}_{XX}(T) = S + T N_0$,该分解表现为广义特征值收缩规则,显式给出温度与可预测容量间的单调权衡。几何上,在白化坐标系中,可预测与创新分量对应互补的协方差椭球,使 $C_{ ext{i}}$ 成为受迹控制的创新预算。当 $C_{ ext{i}}$ 较大时,强制高维创新子空间具有方差下界,在弱混合与反集中假设下,产生大量创新块微分熵和指数级可区分的历史。最后,我们给出信息论下界,表明学习诱导的创新块分布需样本数随有效创新维度增长,支持噪声物理储层的生成实用性。

原文摘要 · Abstract (English)

In noisy physical reservoirs, the classical information-processing capacity $C_{\mathrm{ip}}$ quantifies how well a linear readout can realize tasks measurable from the input history, yet $C_{\mathrm{ip}}$ can be far smaller than the observed rank of the readout covariance. We explain this ``missing capacity'' by introducing the innovation capacity $C_{\mathrm{i}}$, the total capacity allocated to readout components orthogonal to the input filtration (Doob innovations, including input-noise mixing). Using a basis-free Hilbert-space formulation of the predictable/innovation decomposition, we prove the conservation law $C_{\mathrm{ip}}+C_{\mathrm{i}}=\mathrm{rank}(Σ_{XX})\le d$, so predictable and innovation capacities exactly partition the rank of the observable readout dimension covariance $Σ_{XX}\in \mathbb{R}^{\rm d\times d}$. In linear-Gaussian Johnson-Nyquist regimes, $Σ_{XX}(T)=S+T N_0$, the split becomes a generalized-eigenvalue shrinkage rule and gives an explicit monotone tradeoff between temperature and predictable capacity. Geometrically, in whitened coordinates the predictable and innovation components correspond to complementary covariance ellipsoids, making $C_{\mathrm{i}}$ a trace-controlled innovation budget. A large $C_{\mathrm{i}}$ forces a high-dimensional innovation subspace with a variance floor and under mild mixing and anti-concentration assumptions this yields extensive innovation-block differential entropy and exponentially many distinguishable histories. Finally, we give an information-theoretic lower bound showing that learning the induced innovation-block law in total variation requires a number of samples that scales with the effective innovation dimension, supporting the generative utility of noisy physical reservoirs.

信息论动态系统创新容量储层计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。