专为电信网络设计的智能问答系统，准确率提升30%。
TeleOracle: Fine-Tuned Retrieval-Augmented Generation with Long-Context Support for Network
- 用两阶段检索增强生成，结合语义分块和混合搜索
- 在问答任务中准确率达81.20%，比原模型高30%
- 小模型实现大模型表现，适合边缘设备部署
电信行业快速发展，亟需能应对复杂网络与新兴技术的智能系统。尽管大语言模型（LLMs）有潜力，但在电信场景中受限于边缘设备性能与文档不一致问题。为此，我们提出基于Phi-2小语言模型（SLM）的专用电信RAG系统TeleOracle。通过两阶段检索器实现语义分块与关键词+语义混合搜索，提升上下文召回效果；推理时扩展上下文窗口以支持长文本；采用低秩适配（LoRA）实现高效微调。在下游问答任务中，该框架使Phi-2准确率提升30%，达81.20%，且优于更大模型，在上下文忠实度上表现更优。
原文摘要 · Abstract (English)
The telecommunications industry's rapid evolution demands intelligent systems capable of managing complex networks and adapting to emerging technologies. While large language models (LLMs) show promise in addressing these challenges, their deployment in telecom environments faces significant constraints due to edge device limitations and inconsistent documentation. To bridge this gap, we present TeleOracle, a telecom-specialized retrieval-augmented generation (RAG) system built on the Phi-2 small language model (SLM). To improve context retrieval, TeleOracle employs a two-stage retriever that incorporates semantic chunking and hybrid keyword and semantic search. Additionally, we expand the context window during inference to enhance the model's performance on open-ended queries. We also employ low-rank adaption for efficient fine-tuning. A thorough analysis of the model's performance indicates that our RAG framework is effective in aligning Phi-2 to the telecom domain in a downstream question and answer (QnA) task, achieving a 30% improvement in accuracy over the base Phi-2 model, reaching an overall accuracy of 81.20%. Notably, we show that our model not only performs on par with the much larger LLMs but also achieves a higher faithfulness score, indicating higher adherence to the retrieved context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。