arXiv:2503.05200cs.CLcs.AI2025-03被引 22

为开放无线接入网开发专用大模型,提升代码生成与理解能力

ORANSight-2.0: Foundational LLMs for O-RAN

  • 构建基于RAG的指令微调框架,自动生成高质量训练数据
  • 在srsRAN基准上实现70B参数模型的代码生成准确率提升32%
  • 使用18个开源模型覆盖1B-70B参数,减少对闭源模型依赖

尽管大语言模型(LLMs)在医疗、客服和商业营销等领域带来变革性影响,但其在开放无线接入网(O-RAN)中的应用仍受限,主要因缺乏领域专用的基础模型。现有方案多依赖通用大模型,难以应对O-RAN的独特挑战。为此,我们提出ORANSight-2.0(O-RAN洞察),一个面向O-RAN的专用基础大模型计划。该计划基于五个开源框架(Mistral、Qwen、Llama、Phi、Gemma)构建18个模型,参数范围从1B到70B,显著降低对专有闭源模型的依赖,并提升在O-RAN任务中的表现。核心是RANSTRUCT框架,一种基于检索增强生成(RAG)的指令微调方法,通过Mistral-based问题生成器和Qwen-based答案生成器构建高质量指令数据集,再使用QLoRA对18个预训练开源模型进行微调。为评估效果,我们引入srsRANBench,一个针对srsRAN(广泛使用的5G O-RAN栈)代码生成与代码库理解的新基准。

原文摘要 · Abstract (English)

Despite the transformative impact of Large Language Models (LLMs) across critical domains such as healthcare, customer service, and business marketing, their integration into Open Radio Access Networks (O-RAN) remains limited. This gap is primarily due to the absence of domain-specific foundational models, with existing solutions often relying on general-purpose LLMs that fail to address the unique challenges and technical intricacies of O-RAN. To bridge this gap, we introduce ORANSight-2.0 (O-RAN Insights), a pioneering initiative to develop specialized foundational LLMs tailored for O-RAN. Built on 18 models spanning five open-source LLM frameworks -- Mistral, Qwen, Llama, Phi, and Gemma -- ORANSight-2.0 fine-tunes models ranging from 1B to 70B parameters, significantly reducing reliance on proprietary, closed-source models while enhancing performance in O-RAN-specific tasks. At the core of ORANSight-2.0 is RANSTRUCT, a novel Retrieval-Augmented Generation (RAG)-based instruction-tuning framework that employs two LLM agents -- a Mistral-based Question Generator and a Qwen-based Answer Generator -- to create high-quality instruction-tuning datasets. The generated dataset is then used to fine-tune the 18 pre-trained open-source LLMs via QLoRA. To evaluate ORANSight-2.0, we introduce srsRANBench, a novel benchmark designed for code generation and codebase understanding in the context of srsRAN, a widely used 5G O-RAN stack.

大模型O-RAN代码生成RAG

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。