用能量模型统一建模相关知识,提升对话系统知识检索效果
Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systems
- 将多个相关知识视为整体,用能量函数统一评分
- 在知识检索任务中显著优于传统交叉编码器基线
- 适合需要精准知识选择的对话系统场景
检索器是自然语言处理中从知识库中根据上下文获取相关知识的重要组件。在基于知识的对话系统中,给定上下文时可能存在多个相关且相互关联的知识项。然而,现有检索模型通常假设知识项条件独立。为此,我们提出 Entriever,一种基于能量的检索器。Entriever 直接将候选检索结果整体建模,而非单独建模每个知识项,其相关性得分由能量函数定义。我们探索了多种能量函数架构及训练方法,结果表明,Entriever 在知识检索任务中显著优于强基线交叉编码器模型。此外,在半监督训练下,Entriever 能有效为检索到的知识项打分,并显著提升端到端对话系统的性能。
原文摘要 · Abstract (English)
A retriever, which retrieves relevant knowledge pieces from a knowledge base given a context, is an important component in many natural language processing (NLP) tasks. Retrievers have been introduced in knowledge-grounded dialog systems to improve knowledge acquisition. In knowledge-grounded dialog systems, when conditioning on a given context, there may be multiple relevant and correlated knowledge pieces. However, knowledge pieces are usually assumed to be conditionally independent in current retriever models. To address this issue, we propose Entriever, an energy-based retriever. Entriever directly models the candidate retrieval results as a whole instead of modeling the knowledge pieces separately, with the relevance score defined by an energy function. We explore various architectures of energy functions and different training methods for Entriever, and show that Entriever substantially outperforms the strong cross-encoder baseline in knowledge retrieval tasks. Furthermore, we show that in semi-supervised training of knowledge-grounded dialog systems, Entriever enables effective scoring of retrieved knowledge pieces and significantly improves end-to-end performance of dialog systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。