系统梳理指令微调全流程,助你打造更懂人类意图的AI模型。
Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models
- 分三类构建高质量指令数据:专家标注、大模型蒸馏、自进化机制。
- 对比全参与高效微调方法,突出LoRA等技术在效率与复用上的优势。
- 聚焦跨语言多模态评估难题,推荐医疗法律金融等垂直领域基准。
指令微调是使大语言模型对齐人类意图、安全约束及特定领域需求的关键技术。本文全面综述了完整流程,包括(i)数据收集方法,(ii)全参数与参数高效微调策略,(iii)评估协议。数据构建分为三类范式:专家标注、来自更大模型的蒸馏、自改进机制,各自在质量、可扩展性与资源成本间权衡。微调技术涵盖传统监督训练到轻量级方法如低秩适应(LoRA)和前缀微调,强调计算效率与模型复用。我们进一步分析多语言与多模态场景下忠实性、实用性与安全性的评估挑战,指出医疗、法律、金融等领域专用基准的兴起。最后讨论自动数据生成、自适应优化与鲁棒评估框架的前景,主张数据、算法与人类反馈的深度融合是推动指令微调大模型发展的关键。本综述旨在为研究人员与实践者提供实用参考,以设计既有效又可靠对齐人类意图的模型。
原文摘要 · Abstract (English)
Instruction tuning is a pivotal technique for aligning large language models (LLMs) with human intentions, safety constraints, and domain-specific requirements. This survey provides a comprehensive overview of the full pipeline, encompassing (i) data collection methodologies, (ii) full-parameter and parameter-efficient fine-tuning strategies, and (iii) evaluation protocols. We categorized data construction into three major paradigms: expert annotation, distillation from larger models, and self-improvement mechanisms, each offering distinct trade-offs between quality, scalability, and resource cost. Fine-tuning techniques range from conventional supervised training to lightweight approaches, such as low-rank adaptation (LoRA) and prefix tuning, with a focus on computational efficiency and model reusability. We further examine the challenges of evaluating faithfulness, utility, and safety across multilingual and multimodal scenarios, highlighting the emergence of domain-specific benchmarks in healthcare, legal, and financial applications. Finally, we discuss promising directions for automated data generation, adaptive optimization, and robust evaluation frameworks, arguing that a closer integration of data, algorithms, and human feedback is essential for advancing instruction-tuned LLMs. This survey aims to serve as a practical reference for researchers and practitioners seeking to design LLMs that are both effective and reliably aligned with human intentions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。