系统梳理高效微调技术,让大模型轻松适应新任务。
PEFT A2Z: Parameter-Efficient Fine-Tuning Survey for Large Language and Vision Models
- 按添加、选择、重参数等五类框架分类,梳理主流高效微调方法。
- 仅更新少量参数即可达到接近全量微调的性能,节省大量算力。
- 适合想低成本部署大模型的研究者和工程师参考。
大型语言模型(LLMs)和视觉语言模型(VLMs)已推动人工智能在自然语言处理、计算机视觉及多模态学习中的应用。然而,全量微调这些模型成本高昂,需大量计算资源、内存和特定任务数据。参数高效微调(PEFT)成为解决方案,仅更新少量参数即可适配下游任务。本综述全面梳理了PEFT技术,分析其动机、设计原则与有效性。首先讨论传统微调带来的资源与可访问性挑战,指出过拟合、灾难性遗忘与参数低效等问题。随后提出结构化分类体系,将方法分为添加型、选择型、重参数型、混合型与统一型框架,并系统比较其机制与权衡。此外,探讨了PEFT在语言、视觉与生成建模等领域的应用效果,证明其在降低资源消耗的同时保持强性能。还讨论了可扩展性、可解释性与鲁棒性等开放挑战,并建议未来方向如联邦学习、领域自适应与理论基础研究。旨在为理解PEFT及其在大模型实用化、高效化与可持续化中的作用提供统一视角。
原文摘要 · Abstract (English)
Large models such as Large Language Models (LLMs) and Vision Language Models (VLMs) have transformed artificial intelligence, powering applications in natural language processing, computer vision, and multimodal learning. However, fully fine-tuning these models remains expensive, requiring extensive computational resources, memory, and task-specific data. Parameter-Efficient Fine-Tuning (PEFT) has emerged as a promising solution that allows adapting large models to downstream tasks by updating only a small portion of parameters. This survey presents a comprehensive overview of PEFT techniques, focusing on their motivations, design principles, and effectiveness. We begin by analyzing the resource and accessibility challenges posed by traditional fine-tuning and highlight key issues, such as overfitting, catastrophic forgetting, and parameter inefficiency. We then introduce a structured taxonomy of PEFT methods -- grouped into additive, selective, reparameterized, hybrid, and unified frameworks -- and systematically compare their mechanisms and trade-offs. Beyond taxonomy, we explore the impact of PEFT across diverse domains, including language, vision, and generative modeling, showing how these techniques offer strong performance with lower resource costs. We also discuss important open challenges in scalability, interpretability, and robustness, and suggest future directions such as federated learning, domain adaptation, and theoretical grounding. Our goal is to provide a unified understanding of PEFT and its growing role in enabling practical, efficient, and sustainable use of large models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。