针对卢旺达语低资源场景,提出新型检索模型提升RAG准确性与成本效益。
KinyaColBERT: A Lexically Grounded Retrieval Model for Low-Resource Retrieval-Augmented Generation
- 融合词级交互与形态学分词,生成细粒度上下文嵌入。
- 在卢旺达语农业检索任务上超越主流基线与商业API。
- 适合低资源语言场景的可靠、低成本RAG系统部署者。
大型语言模型(LLM)技术正推动聊天机器人和虚拟助手在多个领域的应用。为增强LLM的可信性并避免幻觉问题,检索增强生成(RAG)成为可行方案。在低资源环境下部署可持续RAG系统,高检索准确率既是可用性需求,也是降低成本的关键。通过对卢旺达语数据集的实证评估,我们发现影响高检索准确率的主要瓶颈在于语言覆盖有限和预训练模型子词分词不足。本文提出新型检索器KinyaColBERT,结合两个核心理念:查询与文档间的后期词级交互,以及基于形态学的分词与双层变压器编码。该方法生成具有词汇根基的细粒度且自包含的上下文嵌入。实验结果表明,KinyaColBERT在卢旺达语农业检索基准上优于强基线及领先的商用文本嵌入API。采用此策略,其他低资源环境的实践者不仅能构建可靠的RAG系统,还可实现更优的成本效益。
原文摘要 · Abstract (English)
The recent mainstream adoption of large language model (LLM) technology is enabling novel applications in the form of chatbots and virtual assistants across many domains. With the aim of grounding LLMs in trusted domains and avoiding the problem of hallucinations, retrieval-augmented generation (RAG) has emerged as a viable solution. In order to deploy sustainable RAG systems in low-resource settings, achieving high retrieval accuracy is not only a usability requirement but also a cost-saving strategy. Through empirical evaluations on a Kinyarwanda-language dataset, we find that the most limiting factors in achieving high retrieval accuracy are limited language coverage and inadequate sub-word tokenization in pre-trained language models. We propose a new retriever model, KinyaColBERT, which integrates two key concepts: late word-level interactions between queries and documents, and a morphology-based tokenization coupled with two-tier transformer encoding. This methodology results in lexically grounded contextual embeddings that are both fine-grained and self-contained. Our evaluation results indicate that KinyaColBERT outperforms strong baselines and leading commercial text embedding APIs on a Kinyarwanda agricultural retrieval benchmark. By adopting this retrieval strategy, we believe that practitioners in other low-resource settings can not only achieve reliable RAG systems but also deploy solutions that are more cost-effective.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。