用多任务微调让小检索器适配多个领域，一器多用降本增效
Multi-task retriever fine-tuning for domain-specific and efficient RAG
- 用多领域指令微调小检索编码器，统一服务多种任务
- 在真实企业场景中实现跨领域泛化与未见任务适应
- 无需为每类应用单独部署，显著提升可扩展性
检索增强生成（RAG）已成为大语言模型部署的主流方案，能有效缓解幻觉和信息过时问题。但在实际应用中面临两大挑战：一是检索内容高度领域化，而微调大模型成本过高，更适合微调检索器；二是同一系统需支持多个应用，难以为每个任务部署独立检索器，且各任务需求的数据类型不同。为此，我们提出对小型检索编码器进行多任务指令微调，使其能通用服务于多种领域和任务，实现低成本、高可扩展性和快速响应。实验表明，该编码器在真实企业场景中不仅能在域外设置下良好泛化，还能适应未见过的检索任务。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) has become ubiquitous when deploying Large Language Models (LLMs), as it can address typical limitations such as generating hallucinated or outdated information. However, when building real-world RAG applications, practical issues arise. First, the retrieved information is generally domain-specific. Since it is computationally expensive to fine-tune LLMs, it is more feasible to fine-tune the retriever to improve the quality of the data included in the LLM input. Second, as more applications are deployed in the same real-world system, one cannot afford to deploy separate retrievers. Moreover, these RAG applications normally retrieve different kinds of data. Our solution is to instruction fine-tune a small retriever encoder on a variety of domain-specific tasks to allow us to deploy one encoder that can serve many use cases, thereby achieving low-cost, scalability, and speed. We show how this encoder generalizes to out-of-domain settings as well as to an unseen retrieval task on real-world enterprise use cases.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。