根据用户需求动态选型大模型，兼顾性能、成本与伦理
Dynamic LLM Routing and Selection based on User Preferences: Balancing Performance, Cost, and Ethics
- 基于任务分析与复杂度估算，智能匹配最优模型
- 融合kNN与分层过滤，实现低延迟高效路由
- 适合云平台、个性化服务及合规性要求高的场景
随着GPT4、BART、LLaMA等大语言模型的广泛应用，如何在成本、延迟、准确率和伦理之间平衡，选择最适合特定任务的模型变得愈发重要。考虑到并非所有任务都需超过1000亿参数的模型，本文提出OptiRoute，一种先进的模型路由引擎，可依据用户定义的详细需求，动态选择并路由任务至最优的LLM。OptiRoute同时考虑功能（如准确率、速度、成本）与非功能（如帮助性、无害性、诚实性）指标，通过轻量级任务分析与复杂度估计，从多样化的LLM中高效匹配最适配模型。采用k近邻（kNN）搜索与分层过滤相结合的混合方法，在满足用户优先级的同时最小化计算开销。该系统适用于云端机器学习平台、个性化AI服务及受监管行业中的实时应用。
原文摘要 · Abstract (English)
With the widespread deployment of large language models (LLMs) such as GPT4, BART, and LLaMA, the need for a system that can intelligently select the most suitable model for specific tasks while balancing cost, latency, accuracy, and ethical considerations has become increasingly important. Recognizing that not all tasks necessitate models with over 100 billion parameters, we introduce OptiRoute, an advanced model routing engine designed to dynamically select and route tasks to the optimal LLM based on detailed user-defined requirements. OptiRoute captures both functional (e.g., accuracy, speed, cost) and non-functional (e.g., helpfulness, harmlessness, honesty) criteria, leveraging lightweight task analysis and complexity estimation to efficiently match tasks with the best-fit models from a diverse array of LLMs. By employing a hybrid approach combining k-nearest neighbors (kNN) search and hierarchical filtering, OptiRoute optimizes for user priorities while minimizing computational overhead. This makes it ideal for real-time applications in cloud-based ML platforms, personalized AI services, and regulated industries.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。