为印度语构建专用嵌入模型，提升检索生成效果
DeepRAG: Building a Custom Hindi Embedding Model for Retrieval Augmented Generation from Scratch
- 从零训练专用哈迪语分词器与注意力机制
- 检索精度比通用模型高23%
- 适合低资源语言的RAG系统开发者
本文介绍了DeepRAG项目，我们为印度语构建了一个专用的嵌入模型，用于增强检索生成（RAG）系统。尽管大语言模型在文本生成方面表现优异，但其检索性能仍严重依赖高质量的嵌入向量——而这一点在印度语中长期缺失。为此，我们从零开始构建，收集超过270万条多样化的哈迪语文本，设计了理解哈迪语形态的SentencePiece分词器，开发了具有哈迪语特异性注意力机制的Transformer架构，并采用对比学习进行优化。实验结果超出预期：检索精度相比现有的多语言模型提升了23%。论文详细阐述了方法论，可为低资源语言研究者提供参考。我们还将该嵌入模型集成至LangChain，构建出完整的哈迪语RAG系统，对实际应用具有指导意义。尽管仍有探索空间，本工作填补了哈迪语自然语言处理的关键空白，验证了语言特异性建模的重要性。
原文摘要 · Abstract (English)
In this paper, I present our work on DeepRAG, a specialized embedding model we built specifically for Hindi language in RAG systems. While LLMs have gotten really good at generating text, their performance in retrieval tasks still depends heavily on having quality embeddings - something that's been lacking for Hindi despite being one of the world's most spoken languages. We tackled this by creating embeddings from the ground up rather than just fine-tuning existing models. Our process involved collecting diverse Hindi texts (over 2.7M samples), training a custom SentencePiece tokenizer that actually understands Hindi morphology, designing transformer architecture with Hindi-specific attention mechanisms, and optimizing with contrastive learning. Results were honestly better than I expected - we saw a 23% improvement in retrieval precision compared to the multilingual models everyone's been using. The paper details our methodology, which I think could help others working with low-resource languages where the one-size-fits-all multilingual models fall short. We've also integrated our embeddings with LangChain to build complete Hindi RAG systems, which might be useful for practitioners. While there's still tons more to explore, I believe this work addresses a critical gap for Hindi NLP and demonstrates why language-specific approaches matter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。