让大模型更聪明地决定何时查外部知识
CCSK:Cognitive Convection of Self-Knowledge Based Retrieval Augmentation for Large Language Models
- 用双模块动态判断是否调用外部知识
- 在多个数据集上提升检索效果
- 适合需要精准问答的智能系统
大语言模型在问答任务中的表现因检索增强生成（RAG）技术得到显著提升，该技术引入了外部知识。然而，主要难点在于平衡模型自身的内在知识与外部信息检索（IR）之间的关系。现有基于阈值的方法采用单一维度的静态机制和单一标准，导致在复杂查询下，其检索决策可能与模型输出无关。为缓解此问题，我们提出基于自知认知传导（CCSK）的检索增强方法。不同于传统方法中固定的单一定制激活标准，CCSK通过孪生网络模块和响应质量模型实现动态联合决策。孪生网络计算当前查询与历史查询之间的余弦相似度，响应质量模型则利用LightGBM评估模型输出的质量。最终决策由两个模块输出及多头注意力融合的文本特征共同决定。大量真实数据集实验表明，CCSK显著提升了模型在信息检索方面的有效性。
原文摘要 · Abstract (English)
The performance of large language models (LLMs) in Q&A task increased substantially through Retrieval-Augmented Generation (RAG) which brings in external knowledge. However, the main difficulty lies in balancing the inherent self-knowledge of LLMs with external information retrieval (IR). The current threshold-based methods apply one-dimensional static mechanisms with single criterion. As a result, their IR decisions might be irrelevant to the LLMs' response under difficult queries. To alleviate this problem, we propose Cognitive Convection of Self-Knowledge (CCSK). Different from traditional methods that maintain single fixed IR activation criteria, CCSK implements a dynamic joint decision process via a Siamese Network module and a Response Quality Model. The Siamese Network calculates the cosine similarity between the current query and the historical queries. The Response Quality Model evaluates the responses of LLMs through LightGBM. The final decision of the CCSK is derived from the outputs of the two modules, as well as text features fused using a multi-head attention mechanism. Extensive experiments on real-world datasets show that CCSK significantly enhances the model's effectiveness in information retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。