让大模型学会像人一样多样地预测下一个词。
Learning to vary: Teaching LMs to reproduce human linguistic variability in next-word prediction
- 用多个合理续词对模型进行微调，模拟人类语言多样性。
- 微调后模型在高低变异性语境下都更接近人类的词汇选择分布。
- 适合希望提升生成结果多样性的自然语言生成研究者。
自然语言生成任务天然存在变异性：给定上下文时，多个合理的词语接续均可能成立，这在多人补全同一句子时尤为明显。尽管让语言模型具备群体视角的多元适应能力非常有益，但Ilia和Aziz（2024）发现现有模型难以有效再现这种语言多样性。他们推测这可能源于训练数据中缺乏对这种内在多样性的持续建模。为此，我们探究在每个上下文中使用多个合理词续进行模型微调是否能提升其对人类语言多样性的复现能力。我们采用微调技术对预训练及指令微调模型（如GPT-2和Mistral-7B-IT）进行实验，使用Provo Corpus数据集。评估通过对比微调前后模型与人类实际下一词分布之间的差异（跨上下文），结果显示，多标签微调显著提升了模型在高、低变异性语境下的多样性复现能力。
原文摘要 · Abstract (English)
Natural language generation (NLG) tasks are often subject to inherent variability; e.g. predicting the next word given a context has multiple valid responses, evident when asking multiple humans to complete the task. While having language models (LMs) that are aligned pluralistically, so that they are able to reproduce well the inherent diversity in perspectives of an entire population of interest is clearly beneficial, Ilia and Aziz (2024) show that LMs do not reproduce this type of linguistic variability well. They speculate this inability might stem from the lack of consistent training of LMs with data reflecting this type of inherent variability. As such, we investigate whether training LMs on multiple plausible word continuations per context can improve their ability to reproduce human linguistic variability for next-word prediction. We employ fine-tuning techniques for pre-trained and instruction-tuned models; and demonstrate their potential when fine-tuning GPT-2 and Mistral-7B-IT, using Provo Corpus. Our evaluation, which measures divergence among empirically estimated human and model next-word distributions across contexts before and after fine-tuning, shows that our multi-label fine-tuning improves the LMs' ability to reproduce linguistic variability; both for contexts that admit higher and lower variability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。