arXiv:2607.06079cs.LGcs.NE2026-07

提出低维扰动学习法,让神经网络高效自适应。

Scalable Perturbation Learning for Online Self-Supervised Learning in Echo State Networks

  • 只对输入相关部分扰动,降低高维系统的方差
  • 保持在线学习与自监督,且无规模依赖的误差增长
  • 适合资源受限的实时系统部署

智能系统需在真实约束下既完成任务又可自适应。自监督学习、在线学习和基于扰动的学习在高维系统中常存在矛盾,因扰动导致的方差随扰动变量维度增大而增长。本文聚焦于大容量储备池网络(ESNs),提出一种基于扰动的在线自监督学习规则。该规则通过正交分解自监督损失函数,将目标分为依赖输入的部分与由固定参数决定的冗余部分。仅对输入相关部分进行扰动,有效扰动维度从储备池维度降至输入维度。该方法同时实现自监督适应、在线学习和标量反馈扰动学习,避免了储备池规模相关的方差增长。结果揭示了一种可扩展且硬件兼容的学习设计原则:在线学习应仅限于目标中动态必需的低维部分。

原文摘要 · Abstract (English)

Intelligent systems should not only solve tasks but also adapt under real-world constraints. Autonomous adaptation via self-supervised learning, sequential adaptation via online learning, and memory-efficient implementation via perturbation-based learning are important requirements for such systems. However, these requirements are generally in tension for high-dimensional systems, because perturbation-based learning suffers from variance that grows with the dimension of the perturbed variables. In this study, we focus on echo state networks (ESNs), where this tension naturally arises in large reservoirs. We propose a perturbation-based learning rule for online self-supervised learning in ESNs. The proposed rule is derived from an orthogonal decomposition of the self-supervised learning cost, which separates an input-dependent component from a redundant component determined by the fixed ESN parameters. By perturbing only the input-dependent component, the effective perturbation dimension is reduced from the reservoir dimension to the input dimension. Thus, the proposed method preserves self-supervised adaptation, online learning, and scalar-feedback perturbation learning, while avoiding reservoir-size-dependent variance growth. This suggests a design principle for scalable and hardware-compatible learning: online learning should be restricted to the dynamically necessary low-dimensional component of the objective.

自监督在线学习储备池网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。