arXiv:2603.13394cs.CV2026-03被引 8

用强化学习动态压缩视觉令牌,显著降低大模型推理开销。

Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models

  • 通过语言引导的强化学习,逐步决策剪枝路径。
  • 最多剪掉66.7%视觉令牌,计算量减少54.2%,精度损失仅0.7%。
  • 适合需要高效推理的视觉语言模型应用者。

大型视觉语言模型(LVLMs)因处理海量视觉令牌而产生巨大推理开销。现有方法难以将视觉令牌渐进式缩减建模为具有序列依赖的多步决策过程,且常依赖缺乏自适应优化的手动评分规则。为此,我们提出TPRL,一种基于强化学习的框架,通过语言引导的序列优化直接学习与最终任务性能相关的自适应剪枝轨迹。我们将视觉令牌剪枝建模为具有显式状态转移的序列决策过程,并采用自监督自编码器将视觉令牌压缩为紧凑状态表示,以实现高效策略学习。剪枝策略初始通过示范学习获得,随后使用近端策略优化(PPO)进行微调,联合优化任务准确率与计算效率。实验结果表明,TPRL在推理过程中最多可移除66.7%的视觉令牌,实现高达54.2%的浮点运算量(FLOPs)减少,同时保持平均精度损失仅为0.7%。代码已公开于https://github.com/MagicVicCoder/TPRL。

原文摘要 · Abstract (English)

Large Vision-Language Models (LVLMs) incur substantial inference costs due to the processing of a vast number of visual tokens. Existing methods typically struggle to model progressive visual token reduction as a multi-step decision process with sequential dependencies and often rely on hand-engineered scoring rules that lack adaptive optimization for complex reasoning trajectories. To overcome these limitations, we propose TPRL, a reinforcement learning framework that learns adaptive pruning trajectories through language-guided sequential optimization tied directly to end-task performance. We formulate visual token pruning as a sequential decision process with explicit state transitions and employ a self-supervised autoencoder to compress visual tokens into a compact state representation for efficient policy learning. The pruning policy is initialized through learning from demonstrations and subsequently fine-tuned using Proximal Policy Optimization (PPO) to jointly optimize task accuracy and computational efficiency. Our experimental results demonstrate that TPRL removes up to 66.7\% of visual tokens and achieves up to a 54.2\% reduction in FLOPs during inference while maintaining a near-lossless average accuracy drop of only 0.7\%. Code is released at \href{https://github.com/MagicVicCoder/TPRL}{\textcolor{mypink}{https://github.com/MagicVicCoder/TPRL}}.

视觉语言模型强化学习推理优化令牌压缩

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。