用强化学习让大模型学会智能交通灯控制，决策更可解释。
DGLight: DQN-Guided GRPO Fine-Tuning of Large Language Models for Traffic Signal Control
- 用DQN评估语言模型的信号动作，指导优化策略
- 在济南、杭州数据上表现优于其他基于大模型的控制器
- 生成的推理过程可读性强，适合交通系统开发者参考
交通信号控制对缓解拥堵、维持城市出行效率至关重要。本文提出DGLight，一种基于批评者引导的强化学习框架，将预训练大语言模型适配到交通信号控制任务中。DGLight首先训练一个基于CoLight的深度Q网络（DQN）批评者，从结构化交叉口状态中估计交通感知的动作价值；随后冻结该批评者，用于评分语言模型生成的候选动作，并通过组相对策略优化（GRPO）更新策略。新控制器能将交通状态映射为可解释的推理链与信号决策，学习来自每状态的密集监督，而非原始累积环境奖励。在涵盖济南和杭州的交通信号控制基准测试中，DGLight是对比的基于大模型控制器中表现最强的，且与强强化学习基线方法相当，并能良好迁移至未用于训练批评者的城市数据集。定性分析显示模型生成的推理过程具有可解释性，且与所选信号相位一致。项目代码已公开。
原文摘要 · Abstract (English)
Traffic signal control (TSC) plays a central role in reducing congestion and maintaining urban mobility. This dissertation introduces DGLight, a critic-guided reinforcement-learning framework for adapting a pretrained large language model to TSC. DGLight first trains a CoLight-based Deep Q-Network critic to estimate traffic-aware action values from structured intersection states, then uses the frozen critic to score candidate language-model actions and optimize the policy with Group Relative Policy Optimization (GRPO). The resulting controller maps traffic states to interpretable reasoning traces and signal decisions while learning from dense per-state supervision rather than raw cumulative environment rewards. Experiments on TSC benchmarks covering Jinan and Hangzhou show that DGLight is the strongest overall method among the compared LLM-based controllers, remains competitive with strong RL baselines, and transfers well to city datasets not used to fit the critic. Qualitative examples further show that the model's generated reasoning is interpretable and aligned with the chosen signal phase. The project code is available $\href{https://github.com/yyccbb/FYP_LLMTSC}{here}$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。