arXiv:2608.28878eess.SYcs.AI2026-08

用混合离线在线强化学习优化无线资源分配,效果媲美集中式方案。

Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management

论文配图:Hybrid Offline-Online Multi-Agent Decision Transformers for Wireless Resource Management
图 1 · 摘自论文原文
  • 基于决策变换器的多智能体框架,先离线预训练再在线微调。
  • 在动态流量下达到与集中式方法相当的业务质量(QoS)性能。
  • 完全分布式,仅需局部观测和邻近信息交换,适合实际部署。

本文提出一种基于决策变换器的混合离线-在线多智能体强化学习框架。策略首先通过现有策略生成轨迹的监督序列建模进行离线预训练,实现安全且样本高效的初始化;随后在线阶段采用融合批评者引导梯度的混合目标进行微调,使性能超越初始离线策略。为促进稳定迁移与有效协同,框架引入回报加权采样、基于邻居动作的批评者以及邻域相关探索机制。该方法完全分布式,训练与执行均仅依赖局部观测及有限的邻近信息交换。在两种动态流量场景下评估:(i) 联合调度与功率分配,(ii) 协同波束成形。结果表明,所提方法在服务质量(QoS)上可媲美集中式方法;即使在低质量数据集上预训练,在线微调仍能超越初始策略。这为无线资源管理提供了有前景的学习型替代方案。

原文摘要 · Abstract (English)

This paper develops a hybrid offline-online multi-agent reinforcement learning framework based on decision transformers. The policy is first pretrained offline via supervised sequence modeling of trajectories generated by existing policies, providing a safe and sample-efficient initialization. It is then fine-tuned online using a hybrid objective that incorporates critic-guided gradients, enabling performance improvements beyond the offline policy. To facilitate stable offline-to-online transfer and effective multi-agent coordination, the framework incorporates return-weighted sampling, a critic conditioned on neighbors' actions, and neighborhood-correlated exploration. The approach is fully distributed: both training and execution rely only on local observations and limited information exchange among neighboring agents. Evaluations with dynamic traffic arrivals in two settings: (i) joint scheduling and power allocation and (ii) coordinated beamforming, show that the proposed method achieves quality-of-service (QoS) performance comparable to centralized methods. Moreover, when pretrained on lower-quality datasets, online fine-tuning is also observed to surpass the initial offline policy. These results demonstrate a promising learning-based alternative for wireless resource management.

多智能体无线资源管理强化学习决策变换器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。