arXiv:2506.05560cs.CL2025-06

用可解释的规则增强大模型,让其更准回答数据库问题。

Improving LLMs with a knowledge from databases

  • 基于知识模式生成规则集,转为文本注入RAG提升问答能力。
  • 相比ChatGPT和带代理的方案,对结构化数据提问准确率显著提升。
  • 方法可解释、可控制,适合安全敏感场景中的知识增强应用。

大语言模型(LLMs)持续快速进步,检索增强生成(RAG)、智能体与工具等技术广泛应用。工具可查询数据库以回答结构化数据问题或执行统计操作,带来巨大潜力,但也引发安全风险,因指令生成缺乏控制。本文提出一种新方法:利用可解释的机器学习方法——增强关联规则,基于预定义的知识模式生成规则集,并通过规则到文本转换器将其转化为自然语言,嵌入到RAG中用于提升大模型表现。实验对比ChatGPT(含智能体)发现,在基于数据集的问题回答上取得显著性能提升。还测试了不同规则数量策略,结果令人满意。未来可拓展至其他模式、将规则挖掘作为智能体等方向。

原文摘要 · Abstract (English)

Large language models (LLMs) are achieving significant progress almost every moment now. Many advanced techniques have been introduced and widely accepted, like retrieval-augmentation generation (RAG), agents, and tools. Tools can query the database to answer questions from structured data files or perform groupings or other statistics. This unlocks huge opportunities, such as it can answer any question, but also poses threats, such as safety, because there is no control over the commands that are created. We would like to discuss whether we can create a new method that improves answers based on dataset/database via some interpretable ML methods, namely enhanced association rules. The advantage would be if the method can be also used in some safe technique like RAG. Association rules have a sound history. Since the introduction of CN2 and aproiri, many enhancements have been made. In parallel, enhanced association rules have been introduced and evolved over the last 40 years. The general problem is typically that there are too many rules. There are some techniques for handling it, but when LLM emerged, it turned out to be the best use case for the RAG technique for LLMs. We proposed a method that generates a ruleset based on defined knowledge patterns, then converts rules into text form via a rule-to-text converter, and includes the result as an RAG into LLM. We compared this method with ChatGPT (even with using agents) and we have discovered a significant improvement in answering questions based on the dataset. We have also tried several strategies how much rules to generate. We found this improvement interesting. Moreover, it can also be improved in many ways as future work, like incorporating other patterns, the use of rule mining as an agent, and many others.

知识增强RAG可解释性规则挖掘

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。