数据少时如何高效微调大模型？这篇综述给出实用方案。
Fine-tuning Large Language Models with Limited Data: A Survey and Practical Guide
- 聚焦参数高效方法，降低训练与部署成本
- 提出在数据稀缺下保持模型性能的适配策略
- 适合资源有限或小语种场景的研究者参考
在低资源语言、专业领域及受限部署环境下，以少量数据微调大型语言模型面临实际挑战。尽管预训练大模型具备强大基础，但在数据稀缺条件下实现有效适配仍需专注且高效的微调技术。本文系统梳理了近期针对数据稀缺场景的微调方法，涵盖参数高效微调技术以降低训练与部署开销，编码器与解码器模型的领域及跨语言适应方法，以及模型专业化策略。进一步考察了利用有限人类或合成反馈进行偏好对齐的方法，强调样本与计算效率。全文突出经验性权衡、选择标准与最佳实践，依据任务约束（如模型规模、数据规模、灾难性遗忘缓解）推荐合适技术。目标是为研究者和实践者提供可操作的洞察，以在数据与资源受限时有效微调大模型。
原文摘要 · Abstract (English)
Fine-tuning large language models (LLMs) with limited data poses a practical challenge in low-resource languages, specialized domains, and constrained deployment settings. While pre-trained LLMs provide strong foundations, effective adaptation under data scarcity requires focused and efficient fine-tuning techniques. This paper presents a structured and practical survey of recent methods for fine-tuning LLMs in data-scarce scenarios. We systematically review parameter-efficient fine-tuning techniques that lower training and deployment costs, domain and cross-lingual adaptation methods for both encoder and decoder models, and model specialization strategies. We further examine preference alignment approaches that guide model behavior using limited human or synthetic feedback, emphasizing sample and compute efficiency. Throughout, we highlight empirical trade-offs, selection criteria, and best practices for choosing suitable techniques based on task constraints, including model scaling, data scaling, and the mitigation of catastrophic forgetting. The aim is to equip researchers and practitioners with actionable insights for effectively fine-tuning LLMs when data and resources are limited.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。