系统梳理提示调优方法，帮大模型低成本高效适配新任务。
A Survey on Prompt Tuning
- 通过可学习的连续向量实现模型参数冻结下的高效适配
- 分类总结直接优化与迁移学习两类主流方法，覆盖多种框架设计
- 揭示计算效率与训练稳定性挑战，适合关注轻量化微调的研究者
本文综述提示调优技术，这是一种通过在输入前添加可学习的连续向量来适应语言模型的参数高效方法，同时保持模型参数冻结。现有方法分为两类：直接提示学习和迁移学习。直接提示学习包括通用优化方法、基于编码器的方法、分解策略以及专家混合框架；迁移学习方法则包含通用迁移策略、基于编码器的方法和分解策略。针对每种方法，分析其设计思路、创新点、洞察、优势与不足，并通过可视化对比不同框架。识别出计算效率与训练稳定性方面的挑战，讨论提升训练鲁棒性及拓展应用范围的未来方向。
原文摘要 · Abstract (English)
This survey reviews prompt tuning, a parameter-efficient approach for adapting language models by prepending trainable continuous vectors while keeping the model frozen. We classify existing approaches into two categories: direct prompt learning and transfer learning. Direct prompt learning methods include: general optimization approaches, encoder-based methods, decomposition strategies, and mixture-of-experts frameworks. Transfer learning methods consist of: general transfer approaches, encoder-based methods, and decomposition strategies. For each method, we analyze method designs, innovations, insights, advantages, and disadvantages, with illustrative visualizations comparing different frameworks. We identify challenges in computational efficiency and training stability, and discuss future directions in improving training robustness and broadening application scope.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。