对比九种适配器架构，发现其训练速度远快于微调且性能相当。
Comparative Analysis of Efficient Adapter-Based Fine-Tuning of State-of-the-Art Transformer Models
- 采用九种先进适配器架构，仅更新少量参数实现高效微调。
- 在SuperGLUE和Kaggle任务上，适配器达到接近或优于全模型微调的准确率。
- 适合资源有限但需快速部署NLP模型的开发者参考使用。
本文研究了多种适配器架构在Superglue基准的监督二分类任务以及Kaggle上的新闻类别多分类任务中的有效性。具体比较了DistilBERT、ELECTRA、BART三种Transformer模型，分别采用传统微调和九种当前最先进的适配器架构，在分类性能与时间复杂度上的表现。结果表明，不同适配器架构间存在性能差异，但均能在远低于微调的训练时间内实现可比甚至更优的分类效果。在新任务上也观察到类似结果，进一步验证了适配器作为微调高效灵活替代方案的潜力。本研究为多样化自然语言处理应用中适配器的选择与实现提供了有价值的指导。
原文摘要 · Abstract (English)
In this work, we investigate the efficacy of various adapter architectures on supervised binary classification tasks from the SuperGLUE benchmark as well as a supervised multi-class news category classification task from Kaggle. Specifically, we compare classification performance and time complexity of three transformer models, namely DistilBERT, ELECTRA, and BART, using conventional fine-tuning as well as nine state-of-the-art (SoTA) adapter architectures. Our analysis reveals performance differences across adapter architectures, highlighting their ability to achieve comparable or better performance relative to fine-tuning at a fraction of the training time. Similar results are observed on the new classification task, further supporting our findings and demonstrating adapters as efficient and flexible alternatives to fine-tuning. This study provides valuable insights and guidelines for selecting and implementing adapters in diverse natural language processing (NLP) applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。