将检索增强微调与低秩适配结合，实现轻量高效问答。
Efficient In-Domain Question Answering for Resource-Constrained Environments
- 用LoRA优化检索增强微调，降低参数量和存储开销。
- 在7B模型上达成媲美GPT-3.5的问答性能。
- 适合无网或算力受限场景下的知识密集型问答任务。
检索增强生成（RAG）是将外部知识融入预训练大语言模型以提升问答准确性和相关性的常用方法。然而，在真实应用场景中，提示工程与资源效率仍是构建高效鲁棒RAG系统的重大瓶颈。近期研究显示，通过微调可有效解决这些问题；特别是针对较小的7B模型应用检索增强微调（RAFT），其表现优于使用大型模型如GPT-3.5的RAG方案。将RAFT与参数高效微调（PEFT）技术（如低秩适配，LoRA）结合，有望带来更高效的解决方案，但该方向尚未被探索。本文首次将RAFT与LoRA结合，显著减少微调与存储需求，加快推理速度，同时保持与传统RAG相当的性能。由此形成的计算更高效的RAFT，即CRAFT，特别适用于互联网受限、硬件资源有限的知识密集型问答任务。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) is a common method for integrating external knowledge into pretrained Large Language Models (LLMs) to enhance accuracy and relevancy in question answering (QA) tasks. However, prompt engineering and resource efficiency remain significant bottlenecks in developing optimal and robust RAG solutions for real-world QA applications. Recent studies have shown success in using fine tuning to address these problems; in particular, Retrieval Augmented Fine Tuning (RAFT) applied to smaller 7B models has demonstrated superior performance compared to RAG setups with much larger models such as GPT-3.5. The combination of RAFT with parameter-efficient fine tuning (PEFT) techniques, such as Low-Rank Adaptation (LoRA), promises an even more efficient solution, yet remains an unexplored area. In this work, we combine RAFT with LoRA to reduce fine tuning and storage requirements and gain faster inference times while maintaining comparable RAG performance. This results in a more compute-efficient RAFT, or CRAFT, which is particularly useful for knowledge-intensive QA tasks in resource-constrained environments where internet access may be restricted and hardware resources limited.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。