用合成数据无监督微调嵌入模型，让小模型在专业数据上表现媲美大模型。
CustomIR: Unsupervised Fine-Tuning of Dense Embeddings for Known Document Corpora
- 用大模型生成带真实文档的合成查询对，自动构造训练数据。
- 在企业邮件和消息数据集上，召回率最高提升2.3点，小模型性能逼近大模型。
- 无需人工标注，适合预算有限但需高精度检索的场景。
稠密嵌入模型在现代信息检索中至关重要，尤其在RAG流程中，但其在预训练分布外的专业语料上性能常下降。为此，我们提出CustomIR框架，通过合成生成的查询-文档对，无监督地适配预训练语言嵌入模型至特定领域语料库。CustomIR利用大语言模型（LLM）生成基于目标语料库的多样化查询，并通过LLM验证获取难负样本，避免昂贵的人工标注。在企业邮件与消息数据集上的实验表明，CustomIR持续提升检索效果，小模型在Recall@10上最高提升2.3点。这一性能提升使小模型可媲美更大模型，从而实现更低成本的RAG部署。结果表明，针对性合成微调是一种可扩展且成本高效的领域性能增强策略。
原文摘要 · Abstract (English)
Dense embedding models have become critical for modern information retrieval, particularly in RAG pipelines, but their performance often degrades when applied to specialized corpora outside their pre-training distribution. To address thi we introduce CustomIR, a framework for unsupervised adaptation of pre-trained language embedding models to domain-specific corpora using synthetically generated query-document pairs. CustomIR leverages large language models (LLMs) to create diverse queries grounded in a known target corpus, paired with LLM-verified hard negatives, eliminating the need for costly human annotation. Experiments on enterprise email and messaging datasets show that CustomIR consistently improves retrieval effectiveness with small models gaining up to 2.3 points in Recall@10. This performance increase allows these small models to rival the performance of much larger alternatives, allowing for cheaper RAG deployments. These results highlight that targeted synthetic fine-tuning offers a scalable and cost-efficient strategy for increasing domain-specific performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。