用1000条高质量推理数据,让开源模型学会闭源模型的复杂推理能力。
ReasonBridge: Efficient Reasoning Transfer from Closed to Open-Source Language Models
- 通过分层知识蒸馏,迁移闭源模型的推理策略与执行细节。
- 仅用0.3%额外参数,使开源模型推理性能提升23%。
- 适合想低成本提升开源模型推理能力的研究者与开发者。
近期大语言模型的发展揭示了闭源与开源模型在复杂推理和精准指令遵循任务上的显著性能差距。本文提出ReasonBridge,一种通过新型分层知识蒸馏框架,高效将闭源模型的推理能力迁移至开源模型的方法。我们构建了仅包含1,000条精心筛选的推理轨迹的数据集Reason1K,强调难度、多样性和质量,采用结构化多准则选择算法从多个领域中过滤生成。该迁移学习方法包括:(1) 分层蒸馏过程,同时捕捉战略抽象与战术执行模式;(2) 稀疏推理专用适配器架构,仅需0.3%额外可训练参数;(3) 测试时计算扩展机制,通过引导式推理干预提升表现。综合评估表明,ReasonBridge使开源模型在基准任务上的推理能力提升最高达23%,显著缩小与闭源模型的差距。值得注意的是,增强后的Qwen2.5-14B在MATH500上优于Claude-Sonnet3.5,且在竞赛级AIME问题上表现相当。该方法在多种推理领域和模型架构间具有强泛化能力,为指令遵循类任务提供了高效的样本优化型推理增强方案。
原文摘要 · Abstract (English)
Recent advancements in Large Language Models (LLMs) have revealed a significant performance gap between closed-source and open-source models, particularly in tasks requiring complex reasoning and precise instruction following. This paper introduces ReasonBridge, a methodology that efficiently transfers reasoning capabilities from powerful closed-source to open-source models through a novel hierarchical knowledge distillation framework. We develop a tailored dataset Reason1K with only 1,000 carefully curated reasoning traces emphasizing difficulty, diversity, and quality. These traces are filtered from across multiple domains using a structured multi-criteria selection algorithm. Our transfer learning approach incorporates: (1) a hierarchical distillation process capturing both strategic abstraction and tactical implementation patterns, (2) a sparse reasoning-focused adapter architecture requiring only 0.3% additional trainable parameters, and (3) a test-time compute scaling mechanism using guided inference interventions. Comprehensive evaluations demonstrate that ReasonBridge improves reasoning capabilities in open-source models by up to 23% on benchmark tasks, significantly narrowing the gap with closed-source models. Notably, the enhanced Qwen2.5-14B outperforms Claude-Sonnet3.5 on MATH500 and matches its performance on competition-level AIME problems. Our methodology generalizes effectively across diverse reasoning domains and model architectures, establishing a sample-efficient approach to reasoning enhancement for instruction following.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。