融合微调与检索生成，构建领域问答增强模型
Aggregated Knowledge Model: Enhancing Domain-Specific QA with Fine-Tuned and Retrieval-Augmented Generation Models
- 用聚类整合七种模型答案，提升领域问答准确性
- 在科学实验室数据集上，集成模型性能显著优于单一模型
- 适合需要高精度专业问答的科研机构系统开发
本文针对劳伦斯伯克利国家实验室（LBL）科学信息技术（ScienceIT）领域的封闭域问答需求，提出一种新方法。基于丰富的ScienceIT文档数据，研究对比了两种微调大模型与五种检索增强生成（RAG）模型。通过数据处理将文档转化为结构化三元组，利用AWS Bedrock、GCP PaLM2、Meta LLaMA2、OpenAI GPT-4、Google Gemini-Pro等最新大模型进行分析。引入聚合知识模型（AKM），采用K-means聚类从七个模型输出中选取最具代表性的答案。多指标评估显示，结合微调与RAG策略的AKM显著提升性能，为特定领域问答系统设计提供有效范式。
原文摘要 · Abstract (English)
This paper introduces a novel approach to enhancing closed-domain Question Answering (QA) systems, focusing on the specific needs of the Lawrence Berkeley National Laboratory (LBL) Science Information Technology (ScienceIT) domain. Utilizing a rich dataset derived from the ScienceIT documentation, our study embarks on a detailed comparison of two fine-tuned large language models and five retrieval-augmented generation (RAG) models. Through data processing techniques, we transform the documentation into structured context-question-answer triples, leveraging the latest Large Language Models (AWS Bedrock, GCP PaLM2, Meta LLaMA2, OpenAI GPT-4, Google Gemini-Pro) for data-driven insights. Additionally, we introduce the Aggregated Knowledge Model (AKM), which synthesizes responses from the seven models mentioned above using K-means clustering to select the most representative answers. The evaluation of these models across multiple metrics offers a comprehensive look into their effectiveness and suitability for the LBL ScienceIT environment. The results demonstrate the potential benefits of integrating fine-tuning and retrieval-augmented strategies, highlighting significant performance improvements achieved with the AKM. The insights gained from this study can be applied to develop specialized QA systems tailored to specific domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。