小模型在RAG系统中实现本地化高效生成，无需GPU
Little Brains, Big Feats: Exploring Compact Language Models
- 用RAG框架测试小型语言模型的生成能力
- 小模型可在设备端10秒内完成推理，无需GPU
- 适合移动端或资源受限场景的开发者参考
尽管大模型主导研究前沿，小模型在多个领域仍具重要价值，但关注度不足。本研究考察了小语言模型在检索增强生成（RAG）系统中的生成表现。通过涵盖不同主题和问题类型的开源与专有数据集进行基准测试，结果表明：配备小模型的RAG系统可在设备端直接运行，且推理时间合理，无需依赖任何GPU硬件。实验代码及补充材料可通过GitHub仓库获取：https://github.com/SibNN/SLM-RAG-EVAL。
原文摘要 · Abstract (English)
While large language models have been dominating the research landscape recently, small language models remain highly relevant across various domains; yet, they receive far less attention. In this study, we investigate how smaller language models perform during the generation stage within a Retrieval-Augmented Generation (RAG) system. To benchmark these models effectively, we utilised both open-source and proprietary datasets covering diverse subject areas and question types. Our findings demonstrate that a RAG system with small language models can be executed directly on-device without requiring any GPU hardware within a reasonable time. The experimental code and links to the supplementary materials can be accessed through the GitHub repository: https://github.com/SibNN/SLM-RAG-EVAL.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。