arXiv:2508.19271cs.CLcs.AI2025-08

用可解释的自动机机制替代模糊提示,提升大模型推理稳定性。

Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT

  • 用任务自适应的加权有限自动机替代全局记忆库,实现结构化检索
  • 在TriviaQA、GSM8K、MMLU上均显著提升性能,且结果可复现
  • 适合需要可解释性与跨领域迁移的可信推理场景

基于提示的推理方法如思维链(CoT)和上下文学习(ICL)虽广泛用于激发大语言模型(LLMs)的推理能力,但依赖脆弱且隐式的机制,常因种子、格式或微小提示变化导致输出不一致,难以保证可靠性。相比之下,基于自动机的神经符号框架如RetoMaton通过符号记忆与确定性转移提供更可靠方案。本文将RetoMaton扩展为局部自适应的加权有限自动机(WFA),直接从外部领域语料构建,增强上下文感知的鲁棒检索能力,同时保持符号可追溯性与低推理开销。相比纠缠上下文与记忆的提示方法,该框架利用WFA显式结构实现可验证、模块化的检索行为,更适合领域迁移与互操作。我们在两个预训练模型LLaMA-3.2-1B和Gemma-3-1B-PT上,在TriviaQA(阅读理解)、GSM8K(多步数学)和MMLU(领域知识)三个任务上评估,结果表明,引入本地RetoMaton能持续提升性能,并实现透明可复现的检索动态,推动现代大模型向轻量级、自动机引导的可信符号推理演进。

原文摘要 · Abstract (English)

Prompt-based reasoning strategies such as Chain-of-Thought (CoT) and In-Context Learning (ICL) have become widely used for eliciting reasoning capabilities in large language models (LLMs). However, these methods rely on fragile, implicit mechanisms often yielding inconsistent outputs across seeds, formats, or minor prompt variations making them fundamentally unreliable for tasks requiring stable, interpretable reasoning. In contrast, automata-based neuro-symbolic frameworks like RetoMaton offer a more structured and trustworthy alternative by grounding retrieval in symbolic memory with deterministic transitions. In this work, we extend RetoMaton by replacing its global datastore with a local, task-adaptive Weighted Finite Automaton (WFA), constructed directly from external domain corpora. This local automaton structure promotes robust, context-aware retrieval while preserving symbolic traceability and low inference overhead. Unlike prompting, which entangles context and memory in opaque ways, our approach leverages the explicit structure of WFAs to provide verifiable and modular retrieval behavior, making it better suited for domain transfer and interoperability. We evaluate this local RetoMaton variant on two pretrained LLMs LLaMA-3.2-1B and Gemma-3-1B-PT across three reasoning tasks: TriviaQA (reading comprehension), GSM8K (multi-step math), and MMLU (domain knowledge). Compared to the base model and prompting-based methods, augmenting these setups with local RetoMaton consistently improves performance while enabling transparent and reproducible retrieval dynamics. Our results highlight a promising shift toward trustworthy, symbolic reasoning in modern LLMs via lightweight, automaton-guided memory.

符号推理自动机可解释性大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。