提出基于汉明恢复的高效向量符号架构,实现抗噪信息恢复。
Efficient Vector Symbolic Architectures from Histogram Recovery
- 用里德-所罗门与哈达玛码级联构造准正交向量
- 通过直方图恢复算法实现噪声下信息精确重构
- 无需训练即可保证编码、解码与恢复效率
向量符号架构(VSAs)是一类信息表示技术,支持通过绑定和叠加构建复杂结构,并在神经符号人工智能与硬件系统中广泛应用。近期研究提出使用随机线性码,虽能高效解绑且保持准正交性,但其在噪声下难以解码,限制了信息恢复能力。本文利用编码理论工具,证明里德-所罗门码与哈达玛码的级联可生成满足互准正交性的码字。进一步提出‘直方图恢复’问题:给定有限域上的N个直方图,需找到长度为N的里德-所罗门码,使其逐项符号频率符合输入直方图。通过相关列表解码算法给出最优解,并分析其抗噪性能。该方法在不依赖启发式或训练的前提下,实现了编码、准正交性与恢复的严格保障,参数优于哈达玛码等已有方案。
原文摘要 · Abstract (English)
Vector symbolic architectures (VSAs) are a family of information representation techniques which enable composition, i.e., creating complex information structures from atomic vectors via binding and superposition, and have recently found wide ranging applications in various neurosymbolic artificial intelligence (AI) systems and hardware systems. Recently, Raviv proposed the use of random linear codes in VSAs, suggesting that their subcode structure enables efficient unbinding, while preserving the quasi-orthogonality that is necessary for neural processing. Yet, random linear codes are difficult to decode under noise, which severely limits the resulting VSA's ability to support recovery, i.e., the retrieval of information objects and their attributes from a noisy compositional representation. In this work we bridge this gap by utilizing coding theoretic tools. First, we argue that the concatenation of Reed-Solomon and Hadamard codes is suitable for VSA, due to the mutual quasi-orthogonality of the resulting codewords (a folklore result). Second, we show that recovery of the resulting compositional representations can be done by solving a problem we call histogram recovery. In histogram recovery, a collection of $N$ histograms over a finite field is given as input, and one must find a collection of Reed-Solomon codewords of length $N$ whose entry-wise symbol frequencies obey those histograms. We present an optimal solution to the histogram recovery problem by using algorithms related to list-decoding, and analyze the resulting noise resilience. Our results give rise to a noise-resilient VSA with formal guarantees regarding efficient encoding, quasi-orthogonality, and recovery, without relying on any heuristics or training, and while operating at improved parameters relative to similar solutions such as the Hadamard code.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。