用深度强化学习动态调节网络拥塞窗口，显著降低延迟且几乎无丢包。
A Deep Reinforcement Learning-Based TCP Congestion Control Algorithm: Design, Simulation, and Evaluation
- 用DQN根据网络状态实时调整拥塞窗口大小
- 相比TCP Cubic延迟降低46.29%、几乎零丢包
- 适合高延迟、易拥塞的现代网络场景
本文提出一种基于深度强化学习（DRL）的TCP拥塞控制算法，采用深度Q网络（DQN）根据观测到的网络状态动态调整拥塞窗口（cWnd）。该方法在NS-3网络仿真器中通过OpenGym接口进行训练与评估。结果表明，相较于传统TCP New Reno和TCP Cubic，该DRL算法在吞吐量与延迟之间取得更优平衡：与TCP Cubic相比，吞吐量差异不显著（-3.79%，p>0.05），但往返时间（RTT）减少46.29%，且几乎无丢包；相较TCP New Reno，吞吐量提升0.38%，RTT降低32.40%。仿真实验显示，该算法有效缓解了缓冲区膨胀（bufferbloat）问题，同时保持高带宽利用率。研究证明，强化学习可通过学习网络容量而非过度占用，解决现代网络中的复杂拥塞控制难题。
原文摘要 · Abstract (English)
This paper introduces a Deep Reinforcement Learning (DRL) based TCP congestion-control algorithm that uses a Deep Q-Network (DQN) to adapt the congestion window (cWnd) dynamically based on observed network state. The proposed approach utilizes DQNs to optimize the congestion window by observing key network parameters and taking real-time actions. The algorithm is trained and evaluated within the NS-3 network simulator using the OpenGym interface. The results demonstrate that the DRL-based algorithm provides a superior balance between throughput and latency compared to both traditional TCP New Reno and TCP Cubic algorithms. Specifically: Compared to TCP Cubic, the DRL algorithm achieved comparable throughput (statistically insignificant difference of -3.79%, $p>0.05$) while delivering a massive 46.29% reduction in Round-Trip Time (RTT). Furthermore, the DRL agent maintained near-zero packet loss, whereas Cubic suffered from significant buffer overflow. Compared to TCP New Reno, the DRL algorithm achieved comparable throughput (+0.38%) with a 32.40% reduction in RTT. Results from NS-3 simulations indicate that the proposed DRL agent effectively mitigates bufferbloat without compromising bandwidth utilization. This study emphasizes the potential of reinforcement learning techniques for solving complex congestion control problems in modern networks by learning the network capacity rather than saturating it.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。