分布式记忆系统通过路由树通信优化各端点的联想记忆。
Distributed Associative Memory via Online Convex Optimization
- 基于在线凸优化，通过路由树实现多智能体协同更新本地记忆
- 理论证明具亚线性损失，长期表现优于现有在线优化方法
- 适合研究分布式学习与神经网络记忆机制的读者
关联记忆（AM）支持线索-响应回忆，近期研究表明其机制支撑现代神经架构如Transformer的运作。本文研究一种分布式场景：各智能体维护本地AM以回忆自身关联及部分他人信息。提出一种基于路由树通信的分布式在线梯度下降方法，用于优化各端点的本地AM。理论分析表明该方法具有亚线性后悔率，实验显示其在性能上持续优于现有在线优化基线。
原文摘要 · Abstract (English)
An associative memory (AM) enables cue-response recall, and associative memorization has recently been noted to underlie the operation of modern neural architectures such as Transformers. This work addresses a distributed setting where agents maintain a local AM to recall their own associations as well as selective information from others. Specifically, we introduce a distributed online gradient descent method that optimizes local AMs at different agents through communication over routing trees. Our theoretical analysis establishes sublinear regret guarantees, and experiments demonstrate that the proposed protocol consistently outperforms existing online optimization baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。