用强化学习为车联网安全应用智能选网,省下40%带宽成本。
Bandwidth Reservation for Time-Critical Vehicular Applications: A Multi-Operator Environment
- 通过深度强化学习在多运营商中自动选最优价格网络
- 实测相比无策略方案降低40%资源成本
- 结合历史数据与实时观测,适合高可靠车联网场景
现场带宽预留常因带宽可用性不可预测和严格延迟要求,面临价格波动与公平性问题。提前预约可缓解波动并保障关键资源及时访问。在多移动运营商环境下,车辆需为安全关键应用选择性价比高且可靠的资源。本研究旨在通过在多个运营商间寻找最低价格来最小化资源成本。将多运营商场景建模为马尔可夫决策过程(MDP),采用双网络深度Q学习(Dueling Deep Q-Learning)算法。为提升学习效率与稳定性,提出一种区域化方法及贴近真实环境的自适应MDP合成机制。使用时间融合变换器(Temporal Fusion Transformer, TFT)处理时序数据并支持模型训练。同时利用亚马逊竞价价格数据,采用分阶段训练策略:先在合成数据上预训练,再用真实数据微调。该策略使DRL代理能基于历史数据与实时观测做出明智决策。实验结果表明,在复杂环境中,本模型相比无策略场景可实现最高达40%的成本降低。
原文摘要 · Abstract (English)
Onsite bandwidth reservation requests often face challenges such as price fluctuations and fairness issues due to unpredictable bandwidth availability and stringent latency requirements. Requesting bandwidth in advance can mitigate the impact of these fluctuations and ensure timely access to critical resources. In a multi-Mobile Network Operator (MNO) environment, vehicles need to select cost-effective and reliable resources for their safety-critical applications. This research aims to minimize resource costs by finding the best price among multiple MNOs. It formulates multi-operator scenarios as a Markov Decision Process (MDP), utilizing a Deep Reinforcement Learning (DRL) algorithm, specifically Dueling Deep Q-Learning. For efficient and stable learning, we propose a novel area-wise approach and an adaptive MDP synthetic close to the real environment. The Temporal Fusion Transformer (TFT) is used to handle time-dependent data and model training. Furthermore, the research leverages Amazon spot price data and adopts a multi-phase training approach, involving initial training on synthetic data, followed by real-world data. These phases enable the DRL agent to make informed decisions using insights from historical data and real-time observations. The results show that our model leads to significant cost reductions, up to 40%, compared to scenarios without a policy model in such a complex environment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。