用大模型增强的决策变换器,让无人机更省电高效地收集数据。
Large Language Model-Empowered Decision Transformer for UAV-Enabled Data Collection
- 用大模型做骨干,结合价值函数正则化,从低质量数据中学习飞行策略。
- 在仿真中比当前最好方法节能36.7%,且只需少量数据训练。
- 适合需要快速部署、数据少的无人机智能控制场景。
将无人机用于从空间分布设备中可靠且节能地收集数据,在支持多样化物联网应用方面具有巨大潜力。然而,无人机续航和通信范围有限,需智能路径规划。尽管强化学习被广泛研究,但其交互式特性在真实环境中有高成本和风险;离线强化学习虽缓解此问题,仍易训练不稳定且高度依赖专家级数据集。为此,我们提出联合无人机路径规划与资源分配问题以最大化数据采集能效。先将资源分配子问题转化为等价线性规划并以多项式时间复杂度求解最优解。随后,提出一种大语言模型赋能的批评者正则化决策变换器框架(LLM-CRDT),通过引入批评网络正则化变换器训练,融合序列建模能力与基于值函数的引导,实现从次优数据集中学习有效策略。为缓解变换器对大量数据的依赖,采用预训练大语言模型作为变换器主干,并使用参数高效微调方法LoRA,使模型能以小规模数据和低计算开销快速适应无人机控制任务。大量仿真表明,LLM-CRDT优于基准在线与离线强化学习方法,相比当前最先进的决策变换器方法,能效最高提升36.7%。
原文摘要 · Abstract (English)
The deployment of unmanned aerial vehicles (UAVs) for reliable and energy-efficient data collection from spatially distributed devices holds great promise in supporting diverse Internet of Things (IoT) applications. Nevertheless, the limited endurance and communication range of UAVs necessitate intelligent trajectory planning. While reinforcement learning (RL) has been extensively explored for UAV trajectory optimization, its interactive nature entails high costs and risks in real-world environments. Offline RL mitigates these issues but remains susceptible to unstable training and heavily rely on expert-quality datasets. To address these challenges, we formulate a joint UAV trajectory planning and resource allocation problem to maximize energy efficiency of data collection. The resource allocation subproblem is first transformed into an equivalent linear programming formulation and solved optimally with polynomial-time complexity. Then, we propose a large language model (LLM)-empowered critic-regularized decision transformer (DT) framework, termed LLM-CRDT, to learn effective UAV control policies. In LLM-CRDT, we incorporate critic networks to regularize the DT model training, thereby integrating the sequence modeling capabilities of DT with critic-based value guidance to enable learning effective policies from suboptimal datasets. Furthermore, to mitigate the data-hungry nature of transformer models, we employ a pre-trained LLM as the transformer backbone of the DT model and adopt a parameter-efficient fine-tuning strategy, i.e., LoRA, enabling rapid adaptation to UAV control tasks with small-scale dataset and low computational overhead. Extensive simulations demonstrate that LLM-CRDT outperforms benchmark online and offline RL methods, achieving up to 36.7\% higher energy efficiency than the current state-of-the-art DT approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。