arXiv:2507.19372cs.AIcs.LG2025-07

用神经符号方法学习可收敛的符号重写系统,实现强泛化与高效推理。

Learning neuro-symbolic convergent term rewriting systems

  • 基于重写算法设计神经符号架构,实现可学习的符号计算系统。
  • 在4个数学公式简化任务中超越神经基线与GPT-4o,接近o1-preview表现。
  • 支持多任务联合训练,内存效率高、推理速度快,适合复杂符号推理场景。

构建能学习执行符号算法的神经系统是人工智能中的一个开放性难题,尤其在追求强泛化和分布外性能时。本文提出一种通用框架,通过受重写算法启发的神经符号架构学习收敛的项重写系统。我们实现了两种模块化架构:神经重写系统(NRS)与快速神经重写系统(FastNRS)。得益于算法驱动的设计与关键结构,两者均具备分布外泛化能力,且FastNRS在内存效率、训练速度与推理时间上均有显著提升。我们在四个涉及数学公式简化的任务上评估了该架构,并进一步展示了其在多领域联合学习场景中的通用性——单个模型可同时解决多种问题类型。所提系统显著优于两个强基线:专为算法问题设计的神经数据路由器(Neural Data Router)及当前最强通用大模型GPT-4o;此外,在推理基准测试中,其表现匹配或超越OpenAI最新o1-preview模型。

原文摘要 · Abstract (English)

Building neural systems that can learn to execute symbolic algorithms is a challenging open problem in artificial intelligence, especially when aiming for strong generalization and out-of-distribution performance. In this work, we introduce a general framework for learning convergent term rewriting systems using a neuro-symbolic architecture inspired by the rewriting algorithm itself. We present two modular implementations of such architecture: the Neural Rewriting System (NRS) and the Fast Neural Rewriting System (FastNRS). As a result of algorithmic-inspired design and key architectural elements, both models can generalize to out-of-distribution instances, with FastNRS offering significant improvements in terms of memory efficiency, training speed, and inference time. We evaluate both architectures on four tasks involving the simplification of mathematical formulas and further demonstrate their versatility in a multi-domain learning scenario, where a single model is trained to solve multiple types of problems simultaneously. The proposed system significantly outperforms two strong neural baselines: the Neural Data Router, a recent transformer variant specifically designed to solve algorithmic problems, and GPT-4o, one of the most powerful general-purpose large-language models. Moreover, our system matches or outperforms the latest o1-preview model from OpenAI that excels in reasoning benchmarks.

神经符号符号推理重写系统泛化能力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。