提出对比学习惩罚方法，提升文本嵌入模型检索效果
Efficient fine-tuning methodology of text embedding models for information retrieval: contrastive learning penalty (clp)
- 设计新型对比学习惩罚函数，优化训练过程
- 在文档检索任务中显著超越现有方法
- 适合需要高效微调嵌入模型的研究者
文本嵌入模型在自然语言处理中扮演关键角色，尤其在信息检索领域，其重要性因RAG（检索增强生成）的应用而进一步凸显。本研究提出一种高效的微调方法，涵盖数据选择、损失函数与模型架构，旨在提升预训练文本嵌入模型的信息检索性能。特别地，提出一种新型对比学习惩罚（Contrastive Learning Penalty, CLP）函数，克服了现有对比学习的局限性。该方法在文档检索任务中显著优于现有技术。研究成果有望通过嵌入模型的微调，提升信息检索系统性能。代码见 https://github.com/CreaLabs/Enhanced-BGE-M3-with-CLP-and-MoE，最优模型可在 https://huggingface.co/CreaLabs 获取。
原文摘要 · Abstract (English)
Text embedding models play a crucial role in natural language processing, particularly in information retrieval, and their importance is further highlighted with the recent utilization of RAG (Retrieval- Augmented Generation). This study presents an efficient fine-tuning methodology encompassing data selection, loss function, and model architecture to enhance the information retrieval performance of pre-trained text embedding models. In particular, this study proposes a novel Contrastive Learning Penalty function that overcomes the limitations of existing Contrastive Learning. The proposed methodology achieves significant performance improvements over existing methods in document retrieval tasks. This study is expected to contribute to improving the performance of information retrieval systems through fine-tuning of text embedding models. The code for this study can be found at https://github.com/CreaLabs/Enhanced-BGE-M3-with-CLP-and-MoE, and the best-performing model can be found at https://huggingface.co/CreaLabs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。