用大模型生成高质量数据,提升垂直领域检索效果
LMAR: Language Model Augmented Retriever for Domain-specific Knowledge Indexing
- 用大模型自动生成标注数据并验证,保证训练质量
- 在多个领域数据集上超越基线模型,延迟低且硬件要求不高
- 框架通用,可无缝接入新模型和检索架构
检索增强生成(RAG)系统在垂直领域常因预训练嵌入性能下降和大语言模型(LLM)检索器计算成本过高而表现不佳。尽管微调数据增强嵌入模型是可行方向,但其效果受限于高质量训练数据和能保持上下文一致性的分块策略。本文提出LMAR(语言模型增强检索器),一种模型无关的框架,通过结合大模型引导的数据合成、对比嵌入适应与高效文本聚类来解决上述挑战。该框架采用两阶段流程:(1)三元组采样与合成数据增强,利用大模型作为标签生成者与验证者,确保整个流程中监督信号的高保真度。在多个垂直领域基准数据集上的实验表明,LMAR优于多种基线模型,同时保持适度的硬件需求与低延迟。其模型无关特性使得其可无缝集成到新兴RAG架构与文本嵌入模型中,实现持续改进而无需重设计流程。结果表明,LMAR是一种实用且低成本的可扩展垂直领域适配方案。
原文摘要 · Abstract (English)
Retrieval Augmented Generation (RAG) systems often struggle with domain-specific knowledge due to performance deterioration of pre-trained embeddings and prohibitive computational costs of large language model (LLM)-based retrievers. While fine-tuning data augmentation embedding models offers a promising direction, its effectiveness is limited by the need for high-quality training data and reliable chunking strategies that preserve contextual integrity. We propose LMAR (Language Model Augmented Retriever), a model-agnostic framework that addresses these challenges by combining LLM-guided data synthesis with contrastive embedding adaptation and efficient text clustering. LMAR consists of a two-stage pipeline: (1) Triplet sampling and synthetic data augmentation, where LLMs act as both labeler and validator to ensure high-fidelity supervision throughout the pipeline. Experimental results across multiple domain-specific benchmark datasets demonstrate that LMAR outperforms multiple baseline models, while maintaining moderate hardware requirements and low latency. Its model-agnostic nature further enables seamless integration with emerging RAG architectures and text embedding models, ensuring continual improvements without redesigning the pipeline. These results highlight LMAR as a practical and cost-effective solution for scalable domain-specific adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。