不用微调，快速找出最适合分类任务的语言模型。
TransformerRanker: A Tool for Efficiently Finding the Best-Suited Language Models for Downstream Classification Tasks
- 基于迁移能力估计方法，不需微调即可排序语言模型。
- 在多个数据集上表现优于现有方法，排名准确率领先。
- 支持HuggingFace生态，适合快速选型的NLP研究者使用。
自然语言处理中的分类任务通常通过从模型库中选择预训练语言模型（PLM）并进行微调来解决。然而，当前可用的PLM数量庞大，如何为特定下游任务选出最优模型成为实际挑战。本文提出TransformerRanker，一个轻量级库，无需计算成本高昂的微调即可高效排名PLM。该库实现了当前主流的可迁移性估计方法（LogME、H-Score、kNN），并结合层聚合策略，经实证表明其排名性能达到最新水平（Garbas et al., 2024）。界面设计简洁，可直接对接HuggingFace Transformers和Datasets库。用户仅需指定下游分类任务和待评估的PLM列表，即可获得最适配模型的排序结果。项目已开源，可通过pip安装：https://github.com/flairNLP/transformer-ranker。
原文摘要 · Abstract (English)
Classification tasks in NLP are typically addressed by selecting a pre-trained language model (PLM) from a model hub, and fine-tuning it for the task at hand. However, given the very large number of PLMs that are currently available, a practical challenge is to determine which of them will perform best for a specific downstream task. With this paper, we introduce TransformerRanker, a lightweight library that efficiently ranks PLMs for classification tasks without the need for computationally costly fine-tuning. Our library implements current approaches for transferability estimation (LogME, H-Score, kNN), in combination with layer aggregation options, which we empirically showed to yield state-of-the-art rankings of PLMs (Garbas et al., 2024). We designed the interface to be lightweight and easy to use, allowing users to directly connect to the HuggingFace Transformers and Dataset libraries. Users need only select a downstream classification task and a list of PLMs to create a ranking of likely best-suited PLMs for their task. We make TransformerRanker available as a pip-installable open-source library https://github.com/flairNLP/transformer-ranker.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。