用全息符号记忆提升大模型信息检索精度
Hypertokens: Holographic Associative Memory in Tokenized LLMs
- 设计超令牌结构，融合纠错码与全息计算实现信息重构
- 在不修改架构前提下，显著提升键值对检索效率
- 适合关注大模型记忆机制优化的研究者
大型语言模型虽具强大能力，却存在信息分散导致的精度损失问题。本文将该问题重新定义为信息理论层面的通信难题。针对模型中关键-值与值-关键的记忆瓶颈，提出HDRAM（全息定义随机存取内存）框架，将Transformer隐空间视为扩频信道。基于超令牌——一种结合经典纠错码、全息计算与量子启发搜索的结构化符号编码——通过有原则的解扩频恢复分布式信息。相位相干的记忆地址支持隐空间中的高效键值操作与格罗弗式搜索。通过将纠错码语法与压缩感知、克里洛夫子空间对齐相结合，HDRAM在不改变原有架构的情况下显著提升关联检索性能，证明了经典-全息-量子启发（CHQ）原理可强化变压器架构。
原文摘要 · Abstract (English)
Large Language Models (LLMs) exhibit remarkable capabilities but suffer from apparent precision loss, reframed here as information spreading. This reframing shifts the problem from computational precision to an information-theoretic communication issue. We address the K:V and V:K memory problem in LLMs by introducing HDRAM (Holographically Defined Random Access Memory), a symbolic memory framework treating transformer latent space as a spread-spectrum channel. Built upon hypertokens, structured symbolic codes integrating classical error-correcting codes (ECC), holographic computing, and quantum-inspired search, HDRAM recovers distributed information through principled despreading. These phase-coherent memory addresses enable efficient key-value operations and Grover-style search in latent space. By combining ECC grammar with compressed sensing and Krylov subspace alignment, HDRAM significantly improves associative retrieval without architectural changes, demonstrating how Classical-Holographic-Quantum-inspired (CHQ) principles can fortify transformer architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。