用逻辑规则增强大模型推理，提升知识注入效率。
RuAG: Learned-rule-augmented Generation for Large Language Models
- 通过自动生成一阶逻辑规则，从离线数据中提炼可解释知识。
- 结合蒙特卡洛树搜索，高效发现高质量逻辑规则并转为自然语言注入。
- 适用于多领域任务，尤其适合需要精准推理的工业场景。
上下文学习（ICL）和检索增强生成（RAG）虽能提升大模型的推理能力，但受限于上下文窗口大小，难以充分注入外部知识。为此，我们提出新框架RuAG，将海量离线数据自动提炼为可解释的一阶逻辑规则，并注入大模型以增强其推理能力。该方法首先利用大模型的常识能力，自动定义规则的头和体谓词；随后采用蒙特卡洛树搜索（MCTS）解决组合搜索空间问题，高效从数据中发现逻辑规则；最终将规则转化为自然语言，实现针对性知识注入与大模型提示的无缝集成。我们在公开及私有工业任务上进行评估，涵盖自然语言处理、时间序列、决策制定和工业应用，验证了该框架在多种任务中对大模型能力的有效提升。
原文摘要 · Abstract (English)
In-context learning (ICL) and Retrieval-Augmented Generation (RAG) have gained attention for their ability to enhance LLMs' reasoning by incorporating external knowledge but suffer from limited contextual window size, leading to insufficient information injection. To this end, we propose a novel framework, RuAG, to automatically distill large volumes of offline data into interpretable first-order logic rules, which are injected into LLMs to boost their reasoning capabilities. Our method begins by formulating the search process relying on LLMs' commonsense, where LLMs automatically define head and body predicates. Then, RuAG applies Monte Carlo Tree Search (MCTS) to address the combinational searching space and efficiently discover logic rules from data. The resulting logic rules are translated into natural language, allowing targeted knowledge injection and seamless integration into LLM prompts for LLM's downstream task reasoning. We evaluate our framework on public and private industrial tasks, including natural language processing, time-series, decision-making, and industrial tasks, demonstrating its effectiveness in enhancing LLM's capability over diverse tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。