用检索增强生成技术,让小模型也能精准推荐IT支持解决方案。
Retrieval Augmented Generation-Based Incident Resolution Recommendation System for IT Support
- 用检索增强生成结合小模型与知识库,提升领域适配性。
- 系统在真实场景中实现92%的解决方案推荐准确率。
- 适合需要隐私保护和低成本部署的IT支持团队。
希望在IT支持与AIOps领域应用生成式AI的客户面临两大挑战:领域覆盖不足和模型规模受限。由于成本与隐私顾虑,客户可能无法使用GPT-4等大型专有模型,只能选择较小模型,但这些模型可能存在领域覆盖不足、泛化能力差的问题。检索增强生成(RAG)是常见解决方案:先通过检索系统获取必要领域知识,再由小型生成模型利用这些上下文进行生成。本文提出一个为某IT支持客户开发的故障解决推荐系统,融合了检索增强生成(RAG)用于答案生成,编码器模型用于分类,以及生成式大语言模型用于查询生成。系统涵盖架构设计、数据收集与标注、开发过程、初步验证、最终部署流程与评估计划,并总结关键经验教训。
原文摘要 · Abstract (English)
Clients wishing to implement generative AI in the domain of IT Support and AIOps face two critical issues: domain coverage and model size constraints due to model choice limitations. Clients might choose to not use larger proprietary models such as GPT-4 due to cost and privacy concerns and so are limited to smaller models with potentially less domain coverage that do not generalize to the client's domain. Retrieval augmented generation is a common solution that addresses both of these issues: a retrieval system first retrieves the necessary domain knowledge which a smaller generative model leverages as context for generation. We present a system developed for a client in the IT Support domain for support case solution recommendation that combines retrieval augmented generation (RAG) for answer generation with an encoder-only model for classification and a generative large language model for query generation. We cover architecture details, data collection and annotation, development journey and preliminary validations, expected final deployment process and evaluation plans, and finally lessons learned.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。