受生物视觉启发,提出可实时自监督学习的递归视觉网络。
Recurrent Joint Embedding Predictive Architecture with Recurrent Forward Propagation Learning
- 用递归门控结构预测下一个图像块表示,实现自监督学习。
- 理论证明算法避免表征坍塌,且能精确执行梯度下降。
- 适合关注类脑学习机制与高效训练方法的研究者。
传统计算机视觉模型依赖深度前馈网络和大量标注数据离线训练,而生物视觉则使用较浅层的递归网络,通过连续注视图像片段实现实时、无监督学习。本文提出一种受生物视觉启发的视觉网络,采用联合嵌入预测架构结合递归门控电路。网络通过基于历史注视序列预测下一图像块表示来学习,属于自监督范式。我们从数学和实证两方面证明该训练算法避免了表征坍塌问题。此外,提出一种名为‘递归-前向传播’的学习算法,无需生物不合理的反向传播通过时间或内存低效的实时递归学习。数学上证明该算法对一大类递归架构实现精确梯度下降,并在实验中验证其高效学习能力。本文聚焦理论创新,暂未开展下游任务性能评估及与生物视觉表征相似性的分析,留待后续工作。
原文摘要 · Abstract (English)
Conventional computer vision models rely on very deep, feedforward networks processing whole images and trained offline with extensive labeled data. In contrast, biological vision relies on comparatively shallow, recurrent networks that analyze sequences of fixated image patches, learning continuously in real-time without explicit supervision. This work introduces a vision network inspired by these biological principles. Specifically, it leverages a joint embedding predictive architecture incorporating recurrent gated circuits. The network learns by predicting the representation of the next image patch (fixation) based on the sequence of past fixations, a form of self-supervised learning. We show mathematical and empirically that the training algorithm avoids the problem of representational collapse. We also introduce \emph{Recurrent-Forward Propagation}, a learning algorithm that avoids biologically unrealistic backpropagation through time or memory-inefficient real-time recurrent learning. We show mathematically that the algorithm implements exact gradient descent for a large class of recurrent architectures, and confirm empirically that it learns efficiently. This paper focuses on these theoretical innovations and leaves empirical evaluation of performance in downstream tasks, and analysis of representational similarity with biological vision for future work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。