动态分配精度，让大模型推理更省力更高效
Dynamic Mixed-Precision Routing for Efficient Multi-step LLM Interaction
- 按每步需求自动选高/低精度模型，灵活节省算力
- 在ALFWorld和WebShop上比单精度模型快2.3倍，准确率不降
- 适合需要长流程推理的复杂任务部署场景
大型语言模型（LLMs）通过测试时的多步交互与推理，在长周期决策任务中表现优异。尽管从业者普遍认为更高的任务成功率需依赖更大更强的模型，但使用大模型进行多步交互会带来高昂的推理开销。为解决此问题，我们探索在长周期决策过程中使用低精度量化的大模型。基于各交互步骤敏感性不同的观察，提出动态混合精度路由（DMR）框架，可在每一步自适应选择高精度或低精度模型。路由器通过两阶段训练：首先采用基于KL散度的监督学习识别精度敏感步骤，再通过组相对策略优化（GRPO）进一步提升任务成功率。在ALFWorld和WebShop上的实验表明，该方法在准确率-成本权衡上显著优于单一精度基线。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve strong performance in long-horizon decision-making tasks through multi-step interaction and reasoning at test time. While practitioners commonly believe a higher task success rate necessitates the use of a larger and stronger LLM model, multi-step interaction with a large LLM incurs prohibitive inference cost. To address this problem, we explore the use of low-precision quantized LLMs in the long-horizon decision-making process. Based on the observation of diverse sensitivities among interaction steps, we propose Dynamic Mixed-Precision Routing (DMR), a framework that adaptively selects between high-precision and low-precision LLMs at each decision step. The router is trained via a two-stage pipeline, consisting of KL-divergence-based supervised learning that identifies precision-sensitive steps, followed by Group-Relative Policy Optimization (GRPO) to further improve task success rates. Experiments on ALFWorld and WebShop demonstrate that our approach achieves a strong accuracy-cost trade-off over single-precision baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。