将大模型表示为提示语义空间中的线性算子，实现高效可解释的模型选择。
Representing LLMs in Prompt Semantic Task Space
- 不需训练，通过几何计算将模型映射到提示语义空间
- 在模型预测和选择任务中表现优异，尤其擅长新场景泛化
- 适合需要快速评估和比较海量模型的研究者
大型语言模型（LLMs）在各类任务中表现卓越，公开预训练模型库日益丰富，但为特定任务挑选最优模型成为重大挑战。现有方法虽尝试学习模型表示，但存在扩展性差、需昂贵重训练等问题，且生成的表示空间难以理解。本文提出一种无需训练的高效方法，将LLM表示为提示语义任务空间中的线性算子，实现高度可解释的模型表征。该方法采用闭式几何计算，具备极强可扩展性与实时适应能力，可应对动态增长的模型库。我们在模型成功预测和选择任务上验证了该方法，结果达到竞争性或领先水平，尤其在样本外场景表现突出。
原文摘要 · Abstract (English)
Large language models (LLMs) achieve impressive results over various tasks, and ever-expanding public repositories contain an abundance of pre-trained models. Therefore, identifying the best-performing LLM for a given task is a significant challenge. Previous works have suggested learning LLM representations to address this. However, these approaches present limited scalability and require costly retraining to encompass additional models and datasets. Moreover, the produced representation utilizes distinct spaces that cannot be easily interpreted. This work presents an efficient, training-free approach to representing LLMs as linear operators within the prompts' semantic task space, thus providing a highly interpretable representation of the models' application. Our method utilizes closed-form computation of geometrical properties and ensures exceptional scalability and real-time adaptability to dynamically expanding repositories. We demonstrate our approach on success prediction and model selection tasks, achieving competitive or state-of-the-art results with notable performance in out-of-sample scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。