arXiv:2608.25583cs.CL2026-08

用可学习插值融合推理与指令模型,兼顾准确与高效

GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning

论文配图:GRIP: Granular Reward-Guided Parameter Interpolation for Efficient Reasoning
图 1 · 摘自论文原文
  • 对齐架构的两模型间逐模块插值,仅优化插值比例
  • 在保持高准确率的同时,推理速度提升2.3倍以上
  • 适合追求推理效率的部署场景,如实时问答系统

面向问题求解的大型语言模型通常通过生成长思维链实现强性能,但显著增加推理成本与延迟。相反,指令微调模型回答更简洁,但推理能力较弱。这种准确率与效率的矛盾催生了一种轻量级方法,可在不重新训练模型的前提下融合两者优势。本文提出GRIP(Granular Reward-guided Interpolation of Parameters),一种基于奖励信号的参数插值框架。给定两个架构相同的推理模型与指令模型,GRIP为每个模块分配可学习的插值比例,并仅优化这些比例,保持原始模型冻结。插值比例通过奖励信号训练,该信号同时偏好正确且简洁的回答。实验表明,GRIP在准确率-效率权衡上优于固定或搜索式融合基线,并揭示了与高效推理相关的模块级融合模式。

原文摘要 · Abstract (English)

Reasoning-oriented large language models often achieve strong problem-solving performance by generating long chains of thought, but this behavior substantially increases inference cost and latency. In contrast, instruction-tuned models tend to answer more concisely, yet often lack comparable reasoning ability. This accuracy-efficiency mismatch motivates a lightweight approach that combines the strengths of both models without full model retraining. In this paper, we propose GRIP (Granular Reward-guided Interpolation of Parameters), a reward-guided parameter interpolation framework for efficient reasoning. Given a reasoning model and an instruction model with identical architectures, GRIP assigns learnable interpolation ratios to individual modules and optimizes only these ratios while keeping both source models frozen. The interpolation ratios are trained with a reward signal that favors responses that are both correct and concise. Experiments show that GRIP achieves a better accuracy-efficiency trade-off than fixed or search-based merging baselines and further reveals module-wise fusion patterns associated with efficient reasoning.

推理优化参数插值模型融合

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。