通过初始化阶段加噪，显著降低迭代中对噪声的依赖，提升向量符号架构因子分解效率。
On the Role of Noise in Factorizers for Disentangling Distributed Representations
- 仅在重建码本初始化时加噪，减少迭代过程中的噪声需求
- 因子数从2增至4时，性能仍比基线提升50倍以上
- 使数字硬件也能高效实现因子分解器，扩展部署场景
为高效将高维分布式表示分解为基本原子向量，可利用向量符号架构（VSA）的超位置计算能力。但此类因子分解器易陷入极限环现象。以往方法需在迭代解码中持续加噪以缓解此问题。本文提出仅在VSA重建码本初始化阶段加噪，即可有效避免极限环。研究发现，当因子数量从2增至4时，最佳性能由初始化噪声主导转向迭代噪声主导；但无论何种情况，新方法均使操作容量较基线因子分解共振网络提升至少50倍。该方案降低了对模拟存内计算系统的需求，使数字硬件同样具备实现可行性，拓宽了因子分解器的应用范围。代码已开源：https://github.com/IBM/in-memory-factorizer
原文摘要 · Abstract (English)
To efficiently factorize high-dimensional distributed representations to the constituent atomic vectors, one can exploit the compute-in-superposition capabilities of vector-symbolic architectures (VSA). Such factorizers however suffer from the phenomenon of limit cycles. Applying noise during the iterative decoding is one mechanism to address this issue. In this paper, we explore ways to further relax the noise requirement by applying noise only at the time of VSA's reconstruction codebook initialization. While the need for noise during iterations proves analog in-memory computing systems to be a natural choice as an implementation media, the adequacy of initialization noise allows digital hardware to remain equally indispensable. This broadens the implementation possibilities of factorizers. Our study finds that while the best performance shifts from initialization noise to iterative noise as the number of factors increases from 2 to 4, both extend the operational capacity by at least 50 times compared to the baseline factorizer resonator networks. Our code is available at: https://github.com/IBM/in-memory-factorizer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。