让大模型按问题难易自动调节推理深度，又快又准还省 token。
Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning
- 根据问题难度动态调整推理长度，快慢思维灵活切换。
- 在7个基准上准确率提升超10%，token消耗减少32.7%-67.3%。
- 适合需要高效推理的视觉语言模型应用，如智能客服、教育助手。
将强化学习（通常通过GRPO）应用于大视觉语言模型推理时，难以有效扩展推理长度，且普遍生成冗长输出，仅带来微弱准确率提升。为解决该问题，本文提出FAST-GRPO，一种基于问题特征动态自适应推理深度的GRPO变体。通过实证分析，验证了在大视觉语言模型中实现快慢思维的可行性，揭示了响应长度与数据分布对性能的影响。受此启发，引入两个互补指标估算问题难度，指导模型判断何时采用快速或慢速推理。随后，在GRPO算法中融入基于长度的自适应奖励和难度感知的KL散度。在七个推理基准上的实验表明，FAST在准确率上达到当前最优水平，相比基线模型提升超过10%，同时相比先前慢思考方法减少32.7%-67.3%的token使用，有效平衡了推理长度与准确率。
原文摘要 · Abstract (English)
When applying reinforcement learning--typically through GRPO--to large vision-language model reasoning struggles to effectively scale reasoning length or generates verbose outputs across all tasks with only marginal gains in accuracy. To address this issue, we present FAST-GRPO, a variant of GRPO that dynamically adapts reasoning depth based on question characteristics. Through empirical analysis, we establish the feasibility of fast-slow thinking in LVLMs by investigating how response length and data distribution affect performance. Inspired by these observations, we introduce two complementary metrics to estimate the difficulty of the questions, guiding the model to determine when fast or slow thinking is more appropriate. Next, we incorporate adaptive length-based rewards and difficulty-aware KL divergence into the GRPO algorithm. Experiments across seven reasoning benchmarks demonstrate that FAST achieves state-of-the-art accuracy with over 10\% relative improvement compared to the base model, while reducing token usage by 32.7-67.3\% compared to previous slow-thinking approaches, effectively balancing reasoning length and accuracy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。