arXiv:2505.07877cs.NIcs.AI2025-05被引 5

用专业数据微调小模型,让AI更好处理电信网络问题。

Efficient Telecom Specific LLM: TSLAM-Mini with QLoRA and Digital Twin Data

  • 用电信专用数据微调3.8亿参数的小模型,提升专业性能。
  • 在20个核心电信场景中表现超越通用大模型,准确率显著提升。
  • 适合电信运维、AI网络管理等需要专业知识的场景使用。

通用大语言模型虽具备广泛能力,但在实时电信应用中常因专业性不足而表现不佳。本文针对此问题,对NetoAI开发的3.8亿参数因果语言模型TSLAM-Mini(基于Phi-4 Mini Instruct 4B)进行精细微调。采用包含10万条样本的定制化数据集,覆盖网络基础、路由、MPLS、网络安全、自动化、OSS/BSS、RAN、移动核心、卫星通信及伦理AI等20个关键电信场景,数据由NetoAI的DigiTwin平台生成,融合领域专家与权威RFC文档的深度洞察,模拟真实网络动态。通过量化低秩适配(QLoRA)技术实现高效微调,支持资源受限设备部署。引入基于Qwen3-235B-A22B的大模型作为自动评判器,严格评估指令遵循与响应质量。实证表明,TSLAM-Mini在电信任务中表现优异,验证了领域专用数据与参数高效微调方法在智能网络管理中的巨大潜力。

原文摘要 · Abstract (English)

General-purpose large language models (LLMs), despite their broad capabilities accrued from open-world data, frequently exhibit suboptimal performance when confronted with the nuanced and specialized demands inherent in real-time telecommunications applications. This investigation addresses this critical limitation through the meticulous fine-tuning of TSLAM-Mini developed by NetoAI, a compact (3.8-billion parameter) causal language model architecturally derived from Phi-4 Mini Instruct 4B. The fine-tuning regimen leverages a bespoke dataset comprising 100,000 samples, strategically engineered to address 20 pivotal telecommunications use-cases, encompassing domains such as Network Fundamentals, IP Routing, MPLS, Network Security, Automation, OSS/BSS, RAN, Mobile Core, Satellite Communications, and Ethical AI. This dataset was curated utilizing NetoAI's DigiTwin platform, enriched with granular insights from venerated network Subject Matter Experts (SMEs) and authoritative RFC documents, thereby capturing high-fidelity representations of real-world network dynamics through simulations inspired by digital twin paradigms. Employing Quantized Low-Rank Adaptation (QLoRA), a state-of-the-art Parameter Efficient Fine-Tuning (PEFT) technique, we achieved substantial training efficiency and enabled prospective deployment on resource-constrained hardware. A novel evaluation framework, predicated on a high-capacity LLM (Qwen3-235B-A22B) functioning as an automated adjudicator, was instituted to rigorously assess instruction-following fidelity and response quality across the specified telecom use-cases. Empirical results unequivocally demonstrate TSLAM-Mini's superior aptitude in telecom-centric applications, underscoring the profound efficacy of domain-specific datasets and PEFT methodologies for advancing intelligent network management.

电信AI小模型QLoRA数字孪生

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。