在强化学习中动态剪枝,让自动驾驶大模型更轻更快。
Before Parc Fermé: RL-Time Pruning for Efficient Embodied LLMs in Autonomous Driving

- 在强化学习过程中分阶段剪枝,利用闭环反馈优化压缩策略。
- 相比直接选小模型,剪枝后性能提升1.69倍,内存与效率平衡更好。
- 适合资源受限的机器人平台,如Jetson AGX Orin,推理速度提升27%。
具身大语言模型(LLM)日益被用作机器人控制流程中的推理模块以提升人机交互能力,但其内存占用和生成延迟使其难以实现实时部署。剪枝可降低计算成本,但对于经历多阶段训练的控制器,关键问题不仅是剪多少,更是何时剪。本文提出在强化学习阶段进行剪枝的策略BPF(Before Parc Fermé),在模型仍处于闭环行为优化过程中完成压缩,使剪枝决策能结合任务特定监督与闭环反馈。提出两种变体:BPF-RL在强化学习中定期迭代剪枝;BPF-SFT/RL先在监督微调阶段剪枝,再在强化学习阶段持续剪枝直至目标比例。在基于机器人驾驶控制管道RobotxR1上评估,使用LLM-Pruner框架,对比后训练剪枝、带强化学习恢复的剪枝、仅微调阶段剪枝及同系列较小稠密模型。结果表明,BPF在任务性能与内存、吞吐量权衡上表现最佳。压缩大模型时,BPF-SFT/RL相比直接选择同系列小模型,参数减少带来的性能损失更小,实现1.69倍的尺寸-端到端性能比优势;在搭载于目标平台的Jetson AGX Orin上,压缩模型解码吞吐量最高提升27%。
原文摘要 · Abstract (English)
Embodied Large Language Models (LLMs) are increasingly used as reasoning modules in robotic control pipelines to improve human-robot interaction, but their memory and generation latency make real-time deployment difficult. Pruning can reduce these costs, but for controllers that undergo multiple pre- and post-training phases, the crucial question is not only how much to prune, but when pruning should occur. In this work, we propose Before Parc Fermé (BPF), a pruning strategy performed during RL that compresses embodied LLM controllers while they are still being optimized for closed-loop behavior. This allows pruning decisions to account for the task-specific supervision and closed-loop feedback that shape the final controller. We propose two variants: BPF-RL, which performs iterative pruning during RL by removing part of the model at predefined training intervals, and BPF-SFT/RL, which first prunes part of the model structure during SFT and then further compresses it during RL using the same iterative strategy as BPF-RL until the target pruning ratio is reached. We evaluate BPF on RobotxR1, an LLM-based autonomous-driving control pipeline, using an established LLM pruning framework (LLM-Pruner), and compare it against post-training pruning, post-training pruning with RL recovery, SFT-stage pruning, and smaller dense models from the same family. Our results show that BPF provides the best task-performance vs. memory and throughput trade-off among the considered pruning strategies. When compressing the larger RobotxR1 models, BPF-SFT/RL achieves a $1.69\times$ better size-end-to-end performance trade-off than directly selecting a smaller dense model from the same family, measured as removed parameters per lost percentage point of control adaptability. On the Jetson AGX Orin mounted on the target robotic platform, the compact models improve decode throughput by up to $27\%$.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。