让大模型协作补足短板，推理时提升准确率10%。
Speculate, then Collaborate: Fusing Knowledge of Language Models during Decoding
- 用草稿模型生成初稿，再由规则判断是否调用助手模型优化
- 跨领域测试中准确率最高提升10%，且无需重新训练
- 适合需要高精度与可解释性的实际应用部署
大型语言模型（LLMs）在特定领域表现优异，但在其他领域受限于训练数据。通过在推理阶段融合多个模型的互补知识，有望提升其跨领域性能。为此，我们提出一种新型协同推测解码（CoSD）算法，可在不需额外训练的情况下，高效实现大模型的知识融合。CoSD利用草稿模型生成初始序列，并采用易于学习的规则或决策树决定是否调用辅助模型进行优化。该方法不仅增强了知识融合能力，还提升了推理效率，具备跨领域、跨模型的可迁移性，并提供更强的可解释性。实验结果表明，相比现有方法，CoSD在多个基准测试中准确率最高提升10%，为基于大模型的应用提供了可扩展、高效的解决方案。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often excel in specific domains but fall short in others due to the limitations of their training. Thus, enabling LLMs to solve problems collaboratively by integrating their complementary knowledge promises to improve their performance across domains. To realize this potential, we introduce a novel Collaborative Speculative Decoding (CoSD) algorithm that enables efficient LLM knowledge fusion at test time without requiring additional model training. CoSD employs a draft model to generate initial sequences and an easy-to-learn rule or decision tree to decide when to invoke an assistant model to improve these drafts. CoSD not only enhances knowledge fusion but also improves inference efficiency, is transferable across domains and models, and offers greater explainability. Experimental results demonstrate that CoSD improves accuracy by up to 10\% across benchmarks compared to existing methods, providing a scalable and effective solution for LLM-based applications
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。