用响应质量分数提升对话中人设一致性，让模型说话前先评估自己是否符合角色。
Score Before You Speak: Improving Persona Consistency in Dialogue Generation using Response Quality Scores
- 训练时让模型学习响应与质量分数的关联，统一优化响应和质量判断。
- 在PERSONA-CHAT和ConvAI2上，人设一致性指标提升5%~8%，适配不同规模模型。
- 通过名词替换增强数据，用语义相似度估算质量，适合改进现有对话系统。
基于人设的对话生成是构建对话式人工智能的重要里程碑。尽管大语言模型能力不断提升，但有效融合人设一致性仍面临挑战，主要因现有对话数据多样性有限。本文提出新框架SBS（Score-Before-Speaking），在百万级和十亿级参数模型上均优于以往方法。不同于传统方法，SBS将响应生成与相对质量学习统一于一步训练中：训练时让模型将增强后的响应与质量分数相关联，并在推理时利用该知识。采用基于名词的替换进行数据增强，以语义相似度作为响应质量的代理指标。在PERSONA-CHAT和ConvAI2基准数据集上的大量实验表明，基于分数条件的训练使模型能更好捕捉多样化的人设一致对话。消融实验进一步证明，在训练输入中引入分数比传统方式更优。代码与更多细节见https://arpita2512.github.io/score_before_you_speak。
原文摘要 · Abstract (English)
Persona-based dialogue generation is an important milestone towards building conversational artificial intelligence. Despite the ever-improving capabilities of large language models (LLMs), effectively integrating persona fidelity in conversations remains challenging due to the limited diversity in existing dialogue data. We propose a novel framework SBS (Score-Before-Speaking), which outperforms previous methods and yields improvements for both million and billion-parameter models. Unlike previous methods, SBS unifies the learning of responses and their relative quality into a single step. The key innovation is to train a dialogue model to correlate augmented responses with a quality score during training and then leverage this knowledge at inference. We use noun-based substitution for augmentation and semantic similarity-based scores as a proxy for response quality. Through extensive experiments with benchmark datasets (PERSONA-CHAT and ConvAI2), we show that score-conditioned training allows existing models to better capture a spectrum of persona-consistent dialogues. Our ablation studies also demonstrate that including scores in the input prompt during training is superior to conventional training setups. Code and further details are available at https://arpita2512.github.io/score_before_you_speak
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。