arXiv:2504.03931cs.CLcs.AI2025-04综述被引 4

让大模型适应特定领域与任务,解决通用模型在专业场景下的短板。

NAACL2025 Tutorial: Adaptation of Large Language Models

  • 从数据与模型双角度,系统梳理大模型适配方法
  • 提出参数化与半参数化两类核心适配技术体系
  • 适合关注模型落地、领域定制与动态更新的从业者

本教程针对通用大模型在金融、医疗、低资源语言代码生成等专业领域表现不佳,且静态特性难以随世界变化而演进的问题,系统介绍动态、领域特定及任务自适应的大模型适配技术。通用大模型虽具备强泛化能力,但其静态结构和庞大体积限制了实际部署。为此,本教程从数据与模型两个视角出发,阐明适配技术的挑战与评估标准,并将适配方法分为两大类:一是参数化知识适配,通过更新模型参数实现知识迁移;二是半参数化知识适配,利用外部知识或工具(如检索增强生成RAG、基于代理的系统)提升模型能力。此外,还将探讨模型编辑等实时适配技术,支持生产环境中的动态更新。

原文摘要 · Abstract (English)

This tutorial on adaptation of LLMs is designed to address the growing demand for models that go beyond the static capabilities of generic LLMs by providing an overview of dynamic, domain-specific, and task-adaptive LLM adaptation techniques. While general LLMs have demonstrated strong generalization across a variety of tasks, they often struggle to perform well in specialized domains such as finance, healthcare, and code generation for underrepresented languages. Additionally, their static nature limits their ability to evolve with the changing world, and they are often extremely large in size, making them impractical and costly to deploy at scale. As a result, the adaptation of LLMs has drawn much attention since the birth of LLMs and is of core importance, both for industry, which focuses on serving its targeted users, and academia, which can greatly benefit from small but powerful LLMs. To address this gap, this tutorial aims to provide an overview of the LLM adaptation techniques. We start with an introduction to LLM adaptation, from both the data perspective and the model perspective. We then emphasize how the evaluation metrics and benchmarks are different from other techniques. After establishing the problems, we explore various adaptation techniques. We categorize adaptation techniques into two main families. The first is parametric knowledge adaptation, which focuses on updating the parametric knowledge within LLMs. Additionally, we will discuss real-time adaptation techniques, including model editing, which allows LLMs to be updated dynamically in production environments. The second kind of adaptation is semi-parametric knowledge adaptation, where the goal is to update LLM parameters to better leverage external knowledge or tools through techniques like retrieval-augmented generation (RAG) and agent-based systems.

大模型适配领域定制RAG模型编辑

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。