MetaRL在交通信号控制中表现不稳定，可能引发高达22%的误差。
The Distribution Shift Problem in Transportation Networks using Reinforcement Learning and AI
- 用MetaLight评估MetaRL在动态交通中的适应性
- 测试显示部分条件下误差达22%，表现不可靠
- 提醒从业者警惕MetaRL在真实交通场景的可靠性风险
近年来，机器学习与人工智能在智能交通网络中的应用显著增加。强化学习（RL）被证明是交通信号控制的有前景方法。然而，其可靠性面临挑战：输入数据分布随时间动态变化，与训练数据分布不一致，导致训练好的RL代理性能下降。本文评估并分析了前沿的元强化学习方法MetaLight，结果表明，在某些条件下其可取得合理效果，但在其他条件下可能表现不佳，误差最高达22%，说明当前元强化学习方案往往缺乏鲁棒性，甚至带来重大可靠性问题。
原文摘要 · Abstract (English)
The use of Machine Learning (ML) and Artificial Intelligence (AI) in smart transportation networks has increased significantly in the last few years. Among these ML and AI approaches, Reinforcement Learning (RL) has been shown to be a very promising approach by several authors. However, a problem with using Reinforcement Learning in Traffic Signal Control is the reliability of the trained RL agents due to the dynamically changing distribution of the input data with respect to the distribution of the data used for training. This presents a major challenge and a reliability problem for the trained network of AI agents and could have very undesirable and even detrimental consequences if a suitable solution is not found. Several researchers have tried to address this problem using different approaches. In particular, Meta Reinforcement Learning (Meta RL) promises to be an effective solution. In this paper, we evaluate and analyze a state-of-the-art Meta RL approach called MetaLight and show that, while under certain conditions MetaLight can indeed lead to reasonably good results, under some other conditions it might not perform well (with errors of up to 22%), suggesting that Meta RL schemes are often not robust enough and can even pose major reliability problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。