动态调整显存分配，提升长尾延迟下的大模型训练效率
DynaResize: Runtime GPU Reallocation for Disaggregated LLM Post-Training
- 运行时动态切换显卡资源，平衡推理与训练耗时
- 端到端吞吐提升66.5%，总耗时减少33%（优于最优静态配置）
- 适合大规模LLM后训练场景，尤其有长尾延迟的系统
基于强化学习的大模型后训练正越来越多地将推理与训练分离至独立的GPU资源上，但静态显存分配在长尾推理延迟下会导致严重的流水线空洞。我们提出DynaResize，一个运行时显存重分配系统，可在不改变强化学习语义的前提下，动态切换显卡资源以平衡各阶段执行时间。DynaResize将重分配分解为细粒度操作，通过通信器复用、状态边界化暂存和基于迟滞的重分配机制，移除了非启动关键任务对关键路径的影响。实验表明，DynaResize可使端到端吞吐提升66.5%，总执行时间减少33%（相较于最优静态配置），同时隐藏27%的角色切换开销。
原文摘要 · Abstract (English)
RL-based LLM post-training increasingly disaggregates Rollout and Training across separate GPU resources, but static GPU partitioning suffers from severe pipeline bubbles under long-tail rollout latency. We present DynaResize, a runtime GPU reallocation system that dynamically switches GPUs between Rollout and Training to balance stage execution times without changing RL semantics. DynaResize decomposes resizing into fine-grained operations and removes non-startup-critical work from the critical path through communicator reuse, bounded state staging, and hysteresis-based resizing. Experimental results show that DynaResize can improve end-to-end throughput by 66.5% and reduce total execution time by 33% over the optimal static configuration, while hiding 27% of role-switching overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。