arXiv:2608.08069cs.SEcs.AI2026-08

让大模型选型像软件组件选择一样透明可查。

HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection

论文配图:HugSelect: An Explainable Multi-Criteria Decision-Support Framework for foundation-model selection
图 1 · 摘自论文原文
  • 构建7万多模型的知识库,融合功能、质量与社区评价
  • 评分可拆解,覆盖率达91%(家族级)和61%(模型级)
  • 适合需要可解释选型的开发者与工程团队

大模型正作为软件组件被广泛复用,模型选型成为关键的工程决策。现有模型平台多依赖流行度,忽视功能能力、运行约束和社区评价。本文提出HugSelect,一个可解释的模型选型支持框架。通过整合仓库元数据、提取的功能特性及从社区讨论中推导的质量属性,构建了包含71,274个模型的知识库。采用加权求和模型进行排序,并展示各准则得分分解。评估显示:功能特征提取F1为0.801,质量属性映射准确率为0.84;模型级覆盖率@10达0.61,家族级达0.91,推荐质量与四个商用LLM系统相当,且推理过程稳定可追溯。消融实验表明功能特征是检索精度主因,用户初步反馈认为该框架直观易用。

原文摘要 · Abstract (English)

Foundation models are increasingly reused as software components, making model selection a critical software-engineering decision. Current model hubs primarily support discovery through popularity metrics, often neglecting functional capabilities, operational constraints, and community-perceived quality. We argue that foundation-model selection should be treated as an explicit, auditable software-component selection task rather than as keyword search, popularity ranking, or opaque conversational advice. This paper proposes HugSelect, an explainable decision-support framework for foundation-model selection. HugSelect builds a knowledge base of 71,274 models by combining repository metadata, extracted functional capabilities, and perceived quality attributes derived from community discussions into a unified pipeline. It ranks candidate models using a weighted additive model that exposes criterion-level score decompositions. We evaluated HugSelect through pipeline validation, comparative case studies against four commercial LLM-based recommendation systems (44 scenarios), fine-grained ablation, and an exploratory user study (n = 10). Extraction pipelines achieved an F1 score of 0.801 for functional features and an accuracy of 0.84 for quality-attribute mapping. HugSelect achieved a model-level Coverage@10 of 0.61 and family-level Coverage@10 of 0.91, showing recommendation quality comparable to that of the evaluated commercial systems, with no significant overall differences in ranking quality, while providing stable, traceable, and inspectable reasoning. Ablation confirmed that functional features were the main driver of retrieval accuracy, and preliminary user feedback suggests that the framework is useful and intuitive.

模型选型可解释性大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。