用两阶段框架让小模型生成更多样且更优的对话回复
Modeling the One-to-Many Property in Open-Domain Dialogue with LLMs
- 分两步：先生成多样回复，再按偏好选最优
- 小模型在多样性上提升显著，质量最高提高90%
- 专为多回答场景设计数据集，适合对话系统研究者
开放域对话具有一个对话上下文对应多个合理回复的一对多特性。尽管已有研究证明建模该特性能提升回复多样性，但多数基于大语言模型的对话代理并未显式处理此问题。本文通过将开放域对话生成分解为多回复生成（MRG）与基于偏好的选择（PS）两个关键任务，实现对一到多特性的建模：首先为给定对话上下文生成n个语义和词汇上多样且高质量的回复，随后根据人类偏好从中选择一个最优回复。为此，我们构建了o2mDial——一个专为捕捉一到多特性而设计的对话语料库，其中每个上下文均包含多个合理回复。基于o2mDial，我们提出新的上下文学习与指令微调策略，以及针对MRG的新评估指标，并开发了一种基于模型的偏好选择方法。实验证明，将该两阶段框架应用于小型语言模型，可在保持上下文连贯性的前提下显著提升回复多样性，响应质量最高提升90%，使其性能接近大型模型。
原文摘要 · Abstract (English)
Open-domain Dialogue (OD) exhibits a one-to-many (o2m) property, whereby multiple appropriate responses exist for a single dialogue context. Despite prior research showing that modeling this property boosts response diversity, most modern LLM-based dialogue agents do not explicitly do so. In this work, we model the o2m property of OD in LLMs by decomposing OD generation into two key tasks: Multi-Response Generation (MRG) and Preference-based Selection (PS), which entail generating a set of n semantically and lexically diverse high-quality responses for a given dialogue context, followed by selecting a single response based on human preference, respectively. To facilitate MRG and PS, we introduce o2mDial, a dialogue corpus explicitly designed to capture the o2m property by featuring multiple plausible responses for each context. Leveraging o2mDial, we propose new in-context learning and instruction-tuning strategies, as well as novel evaluation metrics for MRG, alongside a model-based approach for PS. Empirical results demonstrate that applying the proposed two-stage framework to smaller LLMs for OD generation enhances overall response diversity while maintaining contextual coherence, improving response quality by up to 90%, bringing them closer to the performance of larger models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。