arXiv:2501.11301cs.CLcs.AI2025-01

用问题匹配问题,精准检索知识库内容,避免幻觉。

Question-to-Question Retrieval for Hallucination-Free Knowledge Access: An Approach for Wikipedia and Wikidata Question Answering

  • 用指令微调大模型生成知识点的多种提问,构建问题向量库。
  • 相关问题间余弦相似度超0.9,实现高精度直接检索。
  • 适合需要快速、可靠答案的问答系统,尤其支持多模态查询。

本文提出一种基于知识库(如Wikipedia和Wikidata)的问答方法,通过‘问题到问题’的匹配与检索实现精准知识获取。不同于传统文档嵌入,该方法利用指令微调的大语言模型为每个逻辑内容单元生成全面的问题集合,并将这些问题进行向量嵌入存储,形成与内容对应的映射关系。用户查询经向量化后,与问题向量库匹配,最高相似度对应的内容直接返回,无需生成答案。实验表明,相关问题对间的余弦相似度大于0.9,实现高精度检索。该方法具备计算高效、响应迅速、可扩展性强等优势。在Wikipedia和Wikidata上验证有效,支持从Wikidata结构化事实中检索多媒体内容,为多模态问答开辟新路径。

原文摘要 · Abstract (English)

This paper introduces an approach to question answering over knowledge bases like Wikipedia and Wikidata by performing "question-to-question" matching and retrieval from a dense vector embedding store. Instead of embedding document content, we generate a comprehensive set of questions for each logical content unit using an instruction-tuned LLM. These questions are vector-embedded and stored, mapping to the corresponding content. Vector embedding of user queries are then matched against this question vector store. The highest similarity score leads to direct retrieval of the associated article content, eliminating the need for answer generation. Our method achieves high cosine similarity ( > 0.9 ) for relevant question pairs, enabling highly precise retrieval. This approach offers several advantages including computational efficiency, rapid response times, and increased scalability. We demonstrate its effectiveness on Wikipedia and Wikidata, including multimedia content through structured fact retrieval from Wikidata, opening up new pathways for multimodal question answering.

知识问答向量检索去幻觉多模态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。