用路由系统让专用模型协作，提升大模型效率与效果。
MoDEM: Mixture of Domain Expert Models
- 用BERT路由将问题分发给医疗、数学等专业模型
- 在多个基准上超越同规模通用模型，性价比更高
- 适合需要高精度垂直领域应用的场景
我们提出一种新方法，通过结合领域提示路由与领域专用模型，提升大语言模型的性能与效率。系统采用基于BERT的路由器，将输入提示分配给最合适的领域专家模型，这些模型专门针对医疗、数学和科学等领域优化。研究显示，该方法在多个基准测试中显著优于同规模通用模型，实现更优的性能-成本比。结果表明，未来大模型的发展可能不再依赖单一巨型通用模型，而是构建由小型专业化模型与智能路由系统组成的生态体系，有望实现更高效的资源利用、更低的计算成本和更高的整体表现。
原文摘要 · Abstract (English)
We propose a novel approach to enhancing the performance and efficiency of large language models (LLMs) by combining domain prompt routing with domain-specialized models. We introduce a system that utilizes a BERT-based router to direct incoming prompts to the most appropriate domain expert model. These expert models are specifically tuned for domains such as health, mathematics and science. Our research demonstrates that this approach can significantly outperform general-purpose models of comparable size, leading to a superior performance-to-cost ratio across various benchmarks. The implications of this study suggest a potential paradigm shift in LLM development and deployment. Rather than focusing solely on creating increasingly large, general-purpose models, the future of AI may lie in developing ecosystems of smaller, highly specialized models coupled with sophisticated routing systems. This approach could lead to more efficient resource utilization, reduced computational costs, and superior overall performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。