证明了多智能体强化学习在交通信号控制中的收敛性，为算法稳定性提供理论支撑。
Convergence of Multiagent Learning Systems for Traffic control
- 采用随机逼近方法分析独立学习者的协作学习动态
- 在特定条件下证明了该多智能体算法可收敛
- 适合关注交通控制算法理论可靠性的研究者
班加罗尔等城市的快速城市化导致严重交通拥堵，高效的交通信号控制（TSC）变得至关重要。多智能体强化学习（MARL）常将每个交通信号视为独立智能体，使用Q-learning进行建模，已成为降低平均通勤延误的有前景策略。尽管先前工作（Prashant L A et al.）已通过实验验证该方法的有效性，但其在交通控制场景下的稳定性与收敛性尚未得到严格理论分析。本文填补了这一空白，聚焦于该多智能体算法的理论基础。我们研究了独立学习者在合作型TSC任务中固有的收敛问题，利用随机逼近方法对学习动态进行形式化分析。主要贡献在于，在给定条件下证明了该交通控制多智能体强化学习算法的收敛性，将其从单智能体异步值迭代的收敛证明扩展至多智能体情形。
原文摘要 · Abstract (English)
Rapid urbanization in cities like Bangalore has led to severe traffic congestion, making efficient Traffic Signal Control (TSC) essential. Multi-Agent Reinforcement Learning (MARL), often modeling each traffic signal as an independent agent using Q-learning, has emerged as a promising strategy to reduce average commuter delays. While prior work Prashant L A et. al has empirically demonstrated the effectiveness of this approach, a rigorous theoretical analysis of its stability and convergence properties in the context of traffic control has not been explored. This paper bridges that gap by focusing squarely on the theoretical basis of this multi-agent algorithm. We investigate the convergence problem inherent in using independent learners for the cooperative TSC task. Utilizing stochastic approximation methods, we formally analyze the learning dynamics. The primary contribution of this work is the proof that the specific multi-agent reinforcement learning algorithm for traffic control is proven to converge under the given conditions extending it from single agent convergence proofs for asynchronous value iteration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。