让语言模型学会理解对话中的回应词和填充词，更像真人聊天。
Investigating the Representation of Backchannels and Fillers in Fine-tuned Language Models
- 用三种微调策略在英日语对话数据上训练模型，保留并标注回应词与填充词。
- 微调后模型对回应词的表征区分度更高，轮廓系数提升，语义差异更明显。
- 生成的对话更接近人类表达，适合想提升对话自然度的研究者和开发者。
回应词和填充词是对话中重要的语言现象，但在现代基于Transformer的语言模型中常被视为‘噪声’而被忽略。本文在英、日语对话语料库上，采用三种微调策略研究这些表达在语言模型中的表征。这些语料库完整保留并标注了回应词与填充词，使我们能评估微调对学习这些表达的影响。通过聚类分析发现，微调模型对回应词与填充词的表征具有更高的轮廓系数，表明其能更好区分不同使用场景下的细微语义差异。同时，结合自然语言生成指标与定性分析，验证了微调模型生成的语句更贴近人类对话风格。结果表明，将通用语言模型转化为具备人类对话能力的模型具有可行性。
原文摘要 · Abstract (English)
Backchannels and fillers are important linguistic expressions in dialogue, but often treated as 'noise' to be bypassed in modern transformer-based language models (LMs). Here, we study how they are represented in LMs using three fine-tuning strategies on three dialogue corpora in English and Japanese, in which backchannels and fillers are both preserved and annotated. This allows us to investigate how fine-tuning can help LMs learn these representations. We first apply clustering analysis to the learnt representation of backchannels and fillers, and find increased silhouette scores in representations from fine-tuned models, which suggests that fine-tuning enables LMs to distinguish the nuanced semantic variation in different backchannel and filler use. We also employ natural language generation metrics and qualitative analyses to verify that utterances produced by fine-tuned LMs resemble those produced by humans more closely. Our findings suggest the potential for transforming general LMs into conversational LMs that can produce human-like language more adequately.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。