小模型在大模型时代如何协作与竞争，助力资源受限场景高效应用
What is the Role of Small Models in the LLM Era: A Survey
- 从协作与竞争双视角分析大模型与小模型的关系
- 揭示小模型在实际应用中的关键价值与未被重视的贡献
- 适合关注算力优化与落地部署的研究者与工程师阅读
大型语言模型（LLMs）在推动通用人工智能（AGI）方面取得显著进展，催生了GPT-4、LLaMA-405B等超大规模模型。然而，模型规模扩大导致计算成本和能耗呈指数级增长，使这些模型对资源有限的学术研究者和企业不具可行性。与此同时，小模型（SMs）在实际场景中被广泛使用，其重要性却被低估。这引发了关于小模型在大模型时代作用的关键问题，但此前研究对此关注不足。本文系统探讨大模型与小模型之间的关系，从协作与竞争两个核心视角出发，旨在为从业者提供洞见，深化对小模型贡献的理解，并促进计算资源的更高效利用。代码已公开于 https://github.com/tigerchen52/role_of_small_models。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have made significant progress in advancing artificial general intelligence (AGI), leading to the development of increasingly large models such as GPT-4 and LLaMA-405B. However, scaling up model sizes results in exponentially higher computational costs and energy consumption, making these models impractical for academic researchers and businesses with limited resources. At the same time, Small Models (SMs) are frequently used in practical settings, although their significance is currently underestimated. This raises important questions about the role of small models in the era of LLMs, a topic that has received limited attention in prior research. In this work, we systematically examine the relationship between LLMs and SMs from two key perspectives: Collaboration and Competition. We hope this survey provides valuable insights for practitioners, fostering a deeper understanding of the contribution of small models and promoting more efficient use of computational resources. The code is available at https://github.com/tigerchen52/role_of_small_models
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。