用神经嵌入解决大数小数计算不稳问题，让数字运算更可靠
Neural Isomorphic Fields: A Transformer-based Algebraic Numerical Embedding
- 用固定长度向量替代原始数值，保持加乘比较的代数结构
- 加法在95%以上测试中表现准确，乘法准确率53%-73%
- 适合需要稳定数值计算的AI系统，如金融、科学建模
神经网络在处理极小或极大数值时常因溢出、下溢和输出不稳定而受限。为缓解此问题，本文提出用数值嵌入向量替代原始值。这些嵌入旨在保留关键代数性质的同时避免数值不稳定性。本文首次提出一种固定长度的数字段嵌入，可在有理数域内保持加法、乘法和比较等代数运算。我们引入新型神经同构场（Neural Isomorphic Field），作为群与域等代数结构的神经抽象，其元素为保持计算中代数结构的嵌入向量。实验表明，加法在身份、封闭性、结合律等关键代数测试中准确率超过95%；而乘法在不同代数性质上准确率介于53%至73%之间。结果凸显模型在加法上的优势，并揭示乘法处理仍需改进。
原文摘要 · Abstract (English)
Neural network models often face challenges when processing very small or very large numbers due to issues such as overflow, underflow, and unstable output variations. To mitigate these problems, we propose using embedding vectors for numbers instead of directly using their raw values. These embeddings aim to retain essential algebraic properties while preventing numerical instabilities. In this paper, we introduce, for the first time, a fixed-length number embedding vector that preserves algebraic operations, including addition, multiplication, and comparison, within the field of rational numbers. We propose a novel Neural Isomorphic Field, a neural abstraction of algebraic structures such as groups and fields. The elements of this neural field are embedding vectors that maintain algebraic structure during computations. Our experiments demonstrate that addition performs exceptionally well, achieving over 95 percent accuracy on key algebraic tests such as identity, closure, and associativity. In contrast, multiplication exhibits challenges, with accuracy ranging from 53 percent to 73 percent across various algebraic properties. These findings highlight the model's strengths in preserving algebraic properties under addition while identifying avenues for further improvement in handling multiplication.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。