用大模型自动生成问答对，提升复杂知识问答能力。
Automatic Dataset Generation for Knowledge Intensive Question Answering Tasks
- 用大模型自动生成基于上下文的问答数据，减少人工标注依赖。
- 自动生成的问答对在逻辑连贯性和事实准确性上优于人工标注数据。
- 适合需要高推理能力的知识密集型问答系统研究者使用。
问答系统需从知识库中检索合适答案，但现有系统在处理复杂推理或实时知识整合时仍存在困难，常借助如检索增强生成（RAG）等技术。然而RAG在处理多源信息间的逻辑关联方面仍面临挑战。本文提出一种通过自动化生成基于上下文的问答对来增强大语言模型在知识密集型问答任务中的表现的新方法。该方法利用大模型生成微调数据，降低对人工标注的依赖，提升模型的理解与推理能力。系统包含自动化问答生成器和模型微调模块，通过困惑度、ROUGE、BLEU和BERTScore进行评估。实验表明，生成的问答对在逻辑连贯性与事实准确性上均优于人工标注数据：Mistral-7b-v0.3在自生成问答对上的BERT F1、BLEU和ROUGE分数分别为0.858、0.172、0.260，而人类标注数据对应分数为0.836、0.083、0.139。
原文摘要 · Abstract (English)
A question-answering (QA) system is to search suitable answers within a knowledge base. Current QA systems struggle with queries requiring complex reasoning or real-time knowledge integration. They are often supplemented with retrieval techniques on a data source such as Retrieval-Augmented Generation (RAG). However, RAG continues to face challenges in handling complex reasoning and logical connections between multiple sources of information. A novel approach for enhancing Large Language Models (LLMs) in knowledge-intensive QA tasks is presented through the automated generation of context-based QA pairs. This methodology leverages LLMs to create fine-tuning data, reducing reliance on human labelling and improving model comprehension and reasoning capabilities. The proposed system includes an automated QA generator and a model fine-tuner, evaluated using perplexity, ROUGE, BLEU, and BERTScore. Comprehensive experiments demonstrate improvements in logical coherence and factual accuracy, with implications for developing adaptable Artificial Intelligence (AI) systems. Mistral-7b-v0.3 outperforms Llama-3-8b with BERT F1, BLEU, and ROUGE scores 0.858, 0.172, and 0.260 of for the LLM generated QA pairs compared to scores of 0.836, 0.083, and 0.139 for the human annotated QA pairs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。