无需基础模型，直接用目标语料适配聊天大模型。
Adapting Chat Language Models Using Only Target Unlabeled Language Data
- 通过注入源聊天模型信息，让目标模型自动生成聊天能力
- 在低资源语言上表现优于传统向量迁移方法
- 适合缺乏标注对话数据的场景，尤其对安全性和指令遵循有提升
词汇扩展（VE）是通过添加新词元并在目标无标签数据上继续预训练，实现大语言模型（LLM）语言适应的标准方法。该方法对基于无标签数据训练的基础模型有效，但对通过有标签对话数据训练的聊天模型存在挑战。直接在目标无标签数据上使用VE进行聊天模型适配可能导致聊天能力遗忘。尽管理想，目标聊天数据常因成本高或不可得而难以获取，尤其是低资源语言；机器翻译替代品也未必有效。此前工作提出使用同一系列的基础模型和聊天模型，先用VE在目标无标签数据上适配基础模型，再通过源基础与聊天模型权重差值生成聊天向量（CV），将其转换为聊天模型。本文提出ElChat，一种无需基础模型即可直接在目标无标签数据上适配聊天模型的新方法。它通过注入源聊天模型信息，激发目标模型的聊天能力。ElChat在目标语言性能和安全性方面更稳健、更具竞争力，且在英文、聊天和指令遵循能力上优于CV方法。
原文摘要 · Abstract (English)
Vocabulary expansion (VE) is the de-facto approach to language adaptation of large language models (LLMs) by adding new tokens and continuing pre-training on target data. While this is effective for base models trained on unlabeled data, it poses challenges for chat models trained to follow instructions through labeled conversation data. Directly adapting the latter with VE on target unlabeled data may result in forgetting chat abilities. While ideal, target chat data is often unavailable or costly to create for low-resource languages, and machine-translated alternatives are not always effective. To address this issue, previous work proposed using a base and chat model from the same family. This method first adapts the base LLM with VE on target unlabeled data and then converts it to a chat model by adding a chat vector (CV) derived from the weight difference between the source base and chat models. We propose ElChat, a new language adaptation method for chat LLMs that adapts a chat model directly on target unlabeled data, without a base model. It elicits chat abilities by injecting information from the source chat model. ElChat offers more robust and competitive target language and safety performance while achieving superior English, chat, and instruction-following abilities compared to CV.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。