通过最小语义单元实现多语言模型动态协作，提升推理能力。
Dynamic Collaboration of Multi-Language Models based on Minimal Complete Semantic Units
- 基于分布距离动态选择最优分词，优化多模型协作
- 在多个基准测试中超越传统方法，效果显著
- 适合需要多语言协同推理的场景
本文研究通过分词级多模型协作提升语言模型的推理能力。方法从多个模型提供的下一个分词分布中选择最优分词，进行自回归推理。与‘模型越多越好’的假设相反，我们提出基于分布距离的动态选择策略（DDS），优化协作过程。为解决多模型协作中的词汇不对齐问题，引入最小完整语义单元（MCSU）概念，该机制简单有效，使不同语言模型在语义空间中实现自然对齐。在多个基准测试上的实验结果表明，该方法具有明显优势。代码将发布于 https://github.com/Fanye12/DDS。
原文摘要 · Abstract (English)
This paper investigates the enhancement of reasoning capabilities in language models through token-level multi-model collaboration. Our approach selects the optimal tokens from the next token distributions provided by multiple models to perform autoregressive reasoning. Contrary to the assumption that more models yield better results, we introduce a distribution distance-based dynamic selection strategy (DDS) to optimize the multi-model collaboration process. To address the critical challenge of vocabulary misalignment in multi-model collaboration, we propose the concept of minimal complete semantic units (MCSU), which is simple yet enables multiple language models to achieve natural alignment within the linguistic space. Experimental results across various benchmarks demonstrate the superiority of our method. The code will be available at https://github.com/Fanye12/DDS.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。