让大模型按需思考,简单问题不耗资源,复杂问题才深度推理。
Think Only When You Need with Large Hybrid-Reasoning Models
- 根据问题难易自动决定是否启动深度思考
- 在保持高准确率的同时减少70%以上推理开销
- 适合需要高效推理的实时应用和边缘设备
近期的大规模推理模型(LRMs)通过在生成最终答案前引入扩展思考过程,显著提升了推理能力,但过长的思考流程带来了巨大的令牌消耗与延迟,对简单问题尤为浪费。本文提出首个能根据用户查询上下文自适应判断是否进行思考的大规模混合推理模型(LHRMs)。通过两阶段训练:先进行混合微调(HFT)作为冷启动,再采用提出的混合组策略优化(HGPO)进行在线强化学习,隐式学习选择合适的思考模式。此外,我们引入混合准确率(Hybrid Accuracy)指标量化模型的混合思考能力。大量实验表明,LHRMs能针对不同难度和类型的查询自适应执行混合思考,在推理与通用能力上优于现有LRMs和LLMs,同时显著提升效率。本工作倡导重新思考扩展思考的适用场景,并为构建混合思考系统提供了坚实起点。
原文摘要 · Abstract (English)
Recent Large Reasoning Models (LRMs) have shown substantially improved reasoning capabilities over traditional Large Language Models (LLMs) by incorporating extended thinking processes prior to producing final responses. However, excessively lengthy thinking introduces substantial overhead in terms of token consumption and latency, which is particularly unnecessary for simple queries. In this work, we introduce Large Hybrid-Reasoning Models (LHRMs), the first kind of model capable of adaptively determining whether to perform thinking based on the contextual information of user queries. To achieve this, we propose a two-stage training pipeline comprising Hybrid Fine-Tuning (HFT) as a cold start, followed by online reinforcement learning with the proposed Hybrid Group Policy Optimization (HGPO) to implicitly learn to select the appropriate thinking mode. Furthermore, we introduce a metric called Hybrid Accuracy to quantitatively assess the model's capability for hybrid thinking. Extensive experimental results show that LHRMs can adaptively perform hybrid thinking on queries of varying difficulty and type. It outperforms existing LRMs and LLMs in reasoning and general capabilities while significantly improving efficiency. Together, our work advocates for a reconsideration of the appropriate use of extended thinking processes and provides a solid starting point for building hybrid thinking systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。