为无线通信设计专用数据集和微调框架,提升大模型性能。
Empowering Large Language Models in Wireless Communication: A Novel Dataset and Fine-Tuning Framework
- 构建多跳问答数据集,覆盖难易不同题型。
- 新方法使模型性能提升1.31%至2.24%。
- 适合无线通信领域研究者使用。
本文构建了一个面向无线通信应用的专用数据集,涵盖从易到难的多跳问题,包括判断题与选择题。通过先进语言模型进行实体抽取与题目生成,并经过严格数据筛选以确保质量与相关性。同时提出基于点式信息量(PVI)的微调方法,理论分析其在量化训练数据信息含量上的有效性,在不同模型上分别实现2.24%和1.31%的性能提升。为验证实际效果,采用多智能体框架生成优化问题摘要与非正交多址接入(NOMA)数学问题求解任务,仿真结果显示摘要任务中ROUGE-L指标提升20.9%。此外,研究了微调大模型的缩放规律及在无线通信中的挑战,为大模型适配该领域提供洞见。本数据集与方法旨在推动大模型在无线通信研究与应用中的发展。
原文摘要 · Abstract (English)
In this work, we develop a specialized dataset aimed at enhancing the evaluation and fine-tuning of large language models (LLMs) specifically for wireless communication applications. The dataset includes a diverse set of multi-hop questions, including true/false and multiple-choice types, spanning varying difficulty levels from easy to hard. By utilizing advanced language models for entity extraction and question generation, rigorous data curation processes are employed to maintain high quality and relevance. Additionally, we introduce a Pointwise V-Information (PVI) based fine-tuning method, providing a detailed theoretical analysis and justification for its use in quantifying the information content of training data with 2.24\% and 1.31\% performance boost for different models compared to baselines, respectively. To demonstrate the effectiveness of the fine-tuned models with the proposed methodologies on practical tasks, we also consider different tasks, including summarizing optimization problems from technical papers and solving the mathematical problems related to non-orthogonal multiple access (NOMA), which are generated by using the proposed multi-agent framework. Simulation results show significant performance gain in summarization tasks with 20.9\% in the ROUGE-L metrics. We also study the scaling laws of fine-tuning LLMs and the challenges LLMs face in the field of wireless communications, offering insights into their adaptation to wireless communication tasks. This dataset and fine-tuning methodology aim to enhance the training and evaluation of LLMs, contributing to advancements in LLMs for wireless communication research and applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。