给大模型加个计算器模块，算术题准确率超98%且不依赖外部工具
IGC: Integrating a Gated Calculator into an LLM to Solve Arithmetic Tasks Reliably and Efficiently
- 在LLM中嵌入门控计算器，直接用GPU模拟计算过程
- 在BigBench算术基准上达98%-99%准确率，超越所有更大模型
- 单次运行完成计算，无中间输出，适合需要可靠算术的场景
解决算术任务是基础能力，但现代大语言模型表现不佳。本文提出集成门控计算器（IGC），让大模型通过在GPU上模拟计算器来执行算术运算。我们对Llama模型进行微调，并在BigBench算术基准上测试，其性能超越当前最优方法，优于所有对比模型，包括规模接近两数量级更大的模型。该方法仅需一次迭代即可完成计算，无需外部工具或生成中间标记。所有算术操作均在模型内部完成，不干扰非算术任务。实验显示，该方法在多次训练和所有子任务上均达到98%至99%的准确率，包括此前未被解决的乘法难题。
原文摘要 · Abstract (English)
Solving arithmetic tasks is a simple and fundamental skill, yet modern Large Language Models (LLMs) have great difficulty with them. We introduce the Integrated Gated Calculator (IGC), a module that enables LLMs to perform arithmetic by emulating a calculator on the GPU. We finetune a Llama model with our module and test it on the BigBench Arithmetic benchmark, where it beats the State of the Art, outperforming all models on the benchmark, including models almost two orders of magnitude larger. Our approach takes only a single iteration to run and requires no external tools. It performs arithmetic operations entirely inside the LLM without the need to produce intermediate tokens. It is computationally efficient, interpretable, and avoids side-effects on tasks that do not require arithmetic operations. It reliably achieves 98\% to 99\% accuracy across multiple training runs and for all subtasks, including the substantially harder subtask of multiplication, which was previously unsolved.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。