提升大模型问答能力，融合知识与可控生成
Prompting is not Enough: Exploring Knowledge Integration and Controllable Generation
- 用检索增强+指令微调融合知识，强化模型理解
- 通过一致性约束实现答案格式可控，准确率显著提升
- 适合需要精准答案格式的开放域问答场景
开放域问答（OpenQA）是自然语言处理的核心任务之一，主要从非结构化文本中提取答案。随着大语言模型（LLM）的快速发展，基于LLM的方法因海量参数带来的涌现理解与回答能力，已超越传统方法。然而，现有方法面临两大挑战：如何有效将知识融入LLM，以及如何在不同任务情境下自适应生成特定格式的答案。为此，我们提出新框架GenKI，通过同时探索知识集成与可控生成来提升OpenQA性能。首先训练密集段落检索模型从知识库中获取相关知识；随后引入新型知识集成模型，在微调阶段将检索知识注入指令，增强模型能力；为进一步实现可控生成，采用微调后的LLM结合基于文本一致性的集成策略，确保连贯性、流畅性及答案格式正确性。在TriviaQA、MSMARCO和CMRC2018三个涵盖多种答案格式的数据集上进行大量实验，结果表明GenKI优于当前主流基线。消融实验揭示：检索知识频率与模型准确回忆能力呈线性关系。代码已开源。
原文摘要 · Abstract (English)
Open-domain question answering (OpenQA) represents a cornerstone in natural language processing (NLP), primarily focused on extracting answers from unstructured textual data. With the rapid advancements in Large Language Models (LLMs), LLM-based OpenQA methods have reaped the benefits of emergent understanding and answering capabilities enabled by massive parameters compared to traditional methods. However, most of these methods encounter two critical challenges: how to integrate knowledge into LLMs effectively and how to adaptively generate results with specific answer formats for various task situations. To address these challenges, we propose a novel framework named GenKI, which aims to improve the OpenQA performance by exploring Knowledge Integration and controllable Generation on LLMs simultaneously. Specifically, we first train a dense passage retrieval model to retrieve associated knowledge from a given knowledge base. Subsequently, we introduce a novel knowledge integration model that incorporates the retrieval knowledge into instructions during fine-tuning to intensify the model. Furthermore, to enable controllable generation in LLMs, we leverage a certain fine-tuned LLM and an ensemble based on text consistency incorporating all coherence, fluency, and answer format assurance. Finally, extensive experiments conducted on the TriviaQA, MSMARCO, and CMRC2018 datasets, featuring diverse answer formats, have demonstrated the effectiveness of GenKI with comparison of state-of-the-art baselines. Moreover, ablation studies have disclosed a linear relationship between the frequency of retrieved knowledge and the model's ability to recall knowledge accurately against the ground truth. Our code of GenKI is available at https://github.com/USTC-StarTeam/GenKI
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。