让手机端小模型与云端大模型协作，大幅降本提效。
Minions: Cost-efficient Collaboration Between On-device and Cloud Language Models
- 云端模型拆解任务为短片段，本地并行处理
- 平均成本降低5.7倍，性能损失不足2.1%
- 适合资源受限但需高精度推理的移动场景
我们研究了一种新兴架构：小型本地语言模型可访问本地数据，并与前沿云端语言模型协作，解决涉及金融、医疗和科学推理的长文档任务。这种本地-远程协同能否在不牺牲质量的前提下降低云端推理成本？首先，我们考察一种简单的对话协议，仅本地模型读取完整上下文，该协议使远程成本降低30.4倍，但性能仅恢复至前沿模型的87%。我们发现该协议存在两大局限：本地模型难以遵循远程模型的多步指令，且无法有效处理长上下文。基于此，我们提出改进协议MinionS，由云端模型将任务分解为更易处理的子任务，分块后在本地并行执行。MinionS平均实现5.7倍的成本节约，同时恢复97.9%的远程模型性能。我们的分析揭示了影响本地-远程系统成本与性能权衡的关键设计因素。
原文摘要 · Abstract (English)
We investigate an emerging setup in which a small, on-device language model (LM) with access to local data communicates with a frontier, cloud-hosted LM to solve real-world tasks involving financial, medical, and scientific reasoning over long documents. Can a local-remote collaboration reduce cloud inference costs while preserving quality? First, we consider a naive collaboration protocol where the local and remote models simply chat back and forth. Because only the local model reads the full context, this protocol achieves a 30.4x reduction in remote costs, but recovers only 87% of the performance of the frontier model. We identify two key limitations of this protocol: the local model struggles to (1) follow the remote model's multi-step instructions and (2) reason over long contexts. Motivated by these observations, we study an extension of this protocol, coined MinionS, in which the remote model decomposes the task into easier subtasks over shorter chunks of the document, that are executed locally in parallel. MinionS reduces costs by 5.7x on average while recovering 97.9% of the performance of the remote model alone. Our analysis reveals several key design choices that influence the trade-off between cost and performance in local-remote systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。