共享骨干模型提升电商领域问答效果
BSharedRAG: Backbone Shared Retrieval-Augmented Generation for the E-commerce Domain
- 用领域数据持续预训练共享骨干模型，再用LoRA分别优化检索与生成
- 检索命中率提升5%-13%，生成质量提高23%（BLEU-3）
- 适合需要实时更新商品信息的电商智能客服场景
在电商等领域，由于存在大量长尾实体和频繁更新的信息，检索增强生成（RAG）系统尤为重要。现有方法通常采用独立的检索与生成模块，难以实现任务间的协同优化。本文提出一种新型共享骨干RAG框架（BSharedRAG）：首先利用领域专用语料库对基础模型进行持续预训练，构建领域专用骨干模型；随后基于该共享骨干，分别训练两个即插即用的低秩适配（LoRA）模块，以分别最小化检索与生成损失。实验结果表明，BSharedRAG在两项数据集上的检索评估中，Hit@3指标分别提升5%和13%，生成评估中BLEU-3指标提升23%。相关代码、模型与数据集已公开于https://bsharedrag.github.io。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) system is important in domains such as e-commerce, which has many long-tail entities and frequently updated information. Most existing works adopt separate modules for retrieval and generation, which may be suboptimal since the retrieval task and the generation task cannot benefit from each other to improve performance. We propose a novel Backbone Shared RAG framework (BSharedRAG). It first uses a domain-specific corpus to continually pre-train a base model as a domain-specific backbone model and then trains two plug-and-play Low-Rank Adaptation (LoRA) modules based on the shared backbone to minimize retrieval and generation losses respectively. Experimental results indicate that our proposed BSharedRAG outperforms baseline models by 5% and 13% in Hit@3 upon two datasets in retrieval evaluation and by 23% in terms of BLEU-3 in generation evaluation. Our codes, models, and dataset are available at https://bsharedrag.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。