反对盲目扩大模型规模，提倡高效精简的LLM设计新范式。
Position: Enough of Scaling LLMs! Lets Focus on Downscaling
- 提出系统性降规模框架，兼顾性能与资源节省
- 强调减少计算开销和环境影响，提升部署可行性
- 适合关注可持续性和边缘部署的研究者与开发者
本文挑战当前以神经网络缩放定律为主导的大型语言模型（LLMs）发展范式，主张转向降规模方向。尽管缩放定律揭示了通过增大模型和数据集规模可提升性能，但该方法存在显著局限：计算效率低下、环境负担重、部署困难。为此，本文提出一个整体性的降规模框架，旨在维持性能的同时大幅降低资源需求。论文阐述了摆脱传统缩放范式的实用策略，倡导一种更可持续、高效且可及的LLM开发路径。
原文摘要 · Abstract (English)
We challenge the dominant focus on neural scaling laws and advocate for a paradigm shift toward downscaling in the development of large language models (LLMs). While scaling laws have provided critical insights into performance improvements through increasing model and dataset size, we emphasize the significant limitations of this approach, particularly in terms of computational inefficiency, environmental impact, and deployment constraints. To address these challenges, we propose a holistic framework for downscaling LLMs that seeks to maintain performance while drastically reducing resource demands. This paper outlines practical strategies for transitioning away from traditional scaling paradigms, advocating for a more sustainable, efficient, and accessible approach to LLM development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。