自动化生成电信领域高质量问答数据,无需人工标注
Think Less, Label Better: Multi-Stage Domain-Grounded Synthetic Data Generation for Fine-Tuning Large Language Models in Telecommunications
- 多阶段RAG流程:检索+生成+精炼,基于知识图谱合成答案
- 用定制RAGAS评分过滤低质样本,确保技术准确性
- 适用于需要专业知识的垂直领域,如电信网络故障排查
大语言模型的成功依赖于大规模、高质量的指令遵循与强化学习数据集。然而,在电信网络故障排查等专业任务中,通过人工标注生成此类数据耗时过长,因准确回答需深厚技术背景和上下文理解。本文提出一种全自动、基于检索增强的合成问答对生成管道。该多阶段框架融合检索器、基础生成器和精炼模型,利用从领域知识图谱中检索的文档合成并优化问答对。为保证数据质量,采用定制化的RAGAS评分机制过滤低质样本,生成适合强化微调(RFT)的高质量数据集。我们在真实电信场景中验证了该方法,聚焦无线接入网(RAN)故障排查。结果表明,该流程可无须人工干预生成复杂且上下文丰富的解决方案计划。本工作为专业领域构建指令与强化学习数据集提供了可扩展方案,显著降低对人工标注的依赖,同时保持高技术保真度。
原文摘要 · Abstract (English)
The success of large language models (LLMs) depends heavily on large-scale, high-quality instruction-following and reinforcement datasets. However, generating such data through human annotation is prohibitively time-consuming particularly for domain-specific tasks like telecom network troubleshooting, where accurate responses require deep technical expertise and contextual understanding. In this paper, we present a fully automated, retrieval-augmented pipeline for generating synthetic question-answer (QA) pairs grounded in structured domain knowledge. Our multi-stage framework integrates a retriever, base generator, and refinement model to synthesize and enhance QA pairs using documents retrieved from a domain-specific knowledge graph. To ensure data quality, we employ customized RAGAS-based scoring to filter low-quality samples, producing a high-quality dataset suitable for reinforcement fine-tuning (RFT). We demonstrate our approach in a real-world telecom scenario focused on radio access network (RAN) troubleshooting. The resulting pipeline generates complex, context-rich troubleshooting solution plans without human intervention. This work offers a scalable solution for building instruction and reinforcement datasets in specialized domains, significantly reducing dependence on manual labeling while maintaining high technical fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。