通过实体匿名化提升大模型对话生成对外部知识的依赖度
Improving LLM's Attachment to External Knowledge In Dialogue Generation Tasks Through Entity Anonymization
- 用实体匿名化方法引导模型关注外部知识图谱
- 在OpenDialKG数据集上显著提升知识依附性
- 适合研究知识增强型对话系统的学者参考
基于知识图谱的对话生成（KG-DG）是一项挑战性任务，要求模型有效整合外部知识。尽管大语言模型（LLMs）在多项NLP任务中表现优异，但在KG-DG中利用外部知识的能力仍待探索。我们发现，即使提供了精准检索的知识图谱，LLMs仍倾向于依赖内部知识，导致与外部知识脱节。为此，我们提出LLM-KAT评估框架以衡量生成回复中的知识依附性，并设计一种简单有效的实体匿名化技术，促进模型更好利用外部知识。在OpenDialKG数据集上的实验表明，该方法显著提升了模型对外部知识的使用程度。
原文摘要 · Abstract (English)
Knowledge graph-based dialogue generation (KG-DG) is a challenging task requiring models to effectively incorporate external knowledge into conversational responses. While large language models (LLMs) have achieved impressive results across various NLP tasks, their ability to utilize external knowledge in KG-DG remains under-explored. We observe that LLMs often rely on internal knowledge, leading to detachment from provided knowledge graphs, even when they are given a flawlessly retrieved knowledge graph. First, we introduce LLM-KAT, an evaluation procedure for measuring knowledge attachment in generated responses. Second, we propose a simple yet effective entity anonymization technique to encourage LLMs to better leverage external knowledge. Experiments on the OpenDialKG dataset demonstrate that our approach improves LLMs' attachment on external knowledge.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。