用新词训练替代微调，高效控制模型行为。
Neologism Learning as a Parameter-Efficient Alternative to Fine-Tuning for Model Steering
- 通过引入新词汇（neologism）实现模型行为引导。
- 相同条件下，新词训练效果优于低秩微调（LoRA）。
- 用户可灵活控制模型，且保留原始默认能力。
在语言建模中，新词（neologism）是为表示原词汇表未包含的概念而训练的新标记。新词可用于引导模型特定行为，例如在提示后添加“请用新词作答”。这种行为控制也可通过微调实现，但成本更高、灵活性更低：训练一个新词仅需 d 个参数，同时允许用户继续使用模型的默认行为。本文在相同数据与超参数设置下，对比了新词学习与低秩适配（LoRA）微调的性能，发现新词方法表现更优。此外，研究还发现模型在被询问新词时，会自发生成自己的新词。
原文摘要 · Abstract (English)
In language modeling, neologisms are new tokens trained to represent a concept not already included in a given model's vocabulary. Neologisms can be used to encourage specific behavior in models, for example by appending prompts with "Give me a neologism answer." Behavioral steering can also be achieved through fine-tuning, albeit with more compute and less flexibility: learning a neologism only trains d parameters and allows the user to still access the model's default behavior. We compare the performance of neologism learning against low-rank adaptation (LoRA) fine-tuning, finding that neologisms outperform fine-tuned models under a matched training setup (same data and hyperparameters). We also investigate self-verbalizations of neologisms, and observe that the model will occasionally make up its own new words when asked about a neologism.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。