让大模型自动选对工具，提升编程任务准确率与效率
Routesplain: Towards Faithful and Intervenable Routing for Software-related Tasks
- 基于查询中的任务、领域等可解释概念做路由决策
- 在8项编程任务中优于单个模型，且成本更低
- 支持人工干预调整路由逻辑，适合需要可控性的场景
大型语言模型（LLMs）已广泛应用于各类软件相关任务，但我们发现其表现存在显著差异。将用户请求路由至合适的模型可提升响应质量并降低成本。现有工作主要聚焦于通用模型的黑箱路由。本文提出Routesplain，首个面向软件相关任务的路由系统，涵盖多语言代码生成与修复、输入输出预测及计算机科学问答。不同于以往方法，Routesplain先从查询中提取人类可读的概念（如任务类型、领域、推理复杂度），仅基于这些概念进行路由，从而提供清晰、可信的决策依据。我们在16个先进LLM上对8类软件任务进行了评估，结果表明Routesplain在准确率和成本方面均优于单个模型，并达到或超越所有黑箱基线；概念级干预也揭示了未来优化方向。
原文摘要 · Abstract (English)
LLMs now tackle a wide range of software-related tasks, yet we show that their performance varies markedly both across and within these tasks. Routing user queries to the appropriate LLMs can therefore help improve response quality while reducing cost. Prior work, however, has focused mainly on general-purpose LLM routing via black-box models. We introduce Routesplain, the first LLM router for software-related tasks, including multilingual code generation and repair, input/output prediction, and computer science QA. Unlike existing routing approaches, Routesplain first extracts human-interpretable concepts from each query (e.g., task, domain, reasoning complexity) and only routes based on these concepts, thereby providing intelligible, faithful rationales. We evaluate Routesplain on 16 state-of-the-art LLMs across eight software-related tasks; Routesplain outperforms individual models both in terms of accuracy and cost, and equals or surpasses all black-box baselines, with concept-level intervention highlighting avenues for further router improvements.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。