让大模型在推理时按词元级调用高精度计算,提升效率与可解释性。
PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
- 分训专家、文本转计算模块与路由器,推理时按词元路由计算与推理
- 在复杂任务中准确率高于微调模型,延迟、耗能、显存使用显著降低
- 适合需高精度科学计算的领域,如工程、金融、科研建模
复杂系统任务需要高精度数值计算支持决策,但现有大语言模型无法将此类计算作为内在且可解释的能力集成。多智能体方法虽可借助外部专家,却带来通信开销与可扩展性瓶颈。为此,我们提出物理隔离专家路由网络(PiERN),一种融合计算与推理的架构。该架构在独立训练文本到计算模块与路由器后,将计算能力内生融入神经网络。推理时,路由器在词元级别调度计算与推理,实现单链思维中的迭代交替。我们在线性与非线性计算-推理任务上评估了PiERN,结果表明其不仅准确率高于直接微调的LLM,且在响应延迟、令牌使用量与GPU能耗方面显著优于主流多智能体方法。PiERN为语言模型与科学系统交互提供了高效、可解释、可扩展的新范式。
原文摘要 · Abstract (English)
Tasks on complex systems require high-precision numerical computation to support decisions, but current large language models (LLMs) cannot integrate such computations as an intrinsic and interpretable capability with existing architectures. Multi-agent approaches can leverage external experts, but inevitably introduce communication overhead and suffer from inefficiency caused by limited scalability. To this end, we propose Physically-isolated Experts Routing Network (PiERN), an architecture for integrating computation and reasoning. Instead of the tool-use workflows or function-calling, PiERN endogenously integrates computational capabilities into neural networks after separately training experts, a text-to-computation module, and a router. At inference, the router directs computation and reasoning at the token level, thereby enabling iterative alternation within a single chain of thought. We evaluate PiERN on representative linear and nonlinear computation-reasoning tasks against LLM finetuning and the multi-agent system approaches. Results show that the PiERN architecture achieves not only higher accuracy than directly finetuning LLMs but also significant improvements in response latency, token usage, and GPU energy consumption compared with mainstream multi-agent approaches. PiERN offers an efficient, interpretable, and scalable paradigm for interfacing language models with scientific systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。