用对比学习让大模型理解对话中回应词的语义与语气关系
Aligning Backchannel and Dialogue Context Representations via Contrastive LLM Fine-Tuning
- 用大模型在对话文本上微调，生成上下文表示
- 学出的嵌入比之前方法提升30%以上，能更好匹配上下文与回应词
- 适合做对话理解、语音交互系统优化的研究者
回应词（如‘yeah’‘mhm’‘right’）是简短、不打断的反馈信号，其词汇形式与语调共同传递语用意义。以往研究多关注回应词出现时机预测，但其词汇-语调形式与语义的关系仍不明确。本文提出两阶段框架：首先在对话转录文本上微调大语言模型，获得丰富上下文表示；其次学习对话上下文与回应词实现的联合嵌入空间。通过三元组相似性判断（语调与跨词汇）和上下文-回应词适配任务评估对齐效果。结果表明，所学投影显著提升上下文-回应词检索性能，优于先前方法；同时发现回应词形式高度依赖长程对话上下文，且学习到的嵌入更贴近人类判断，优于原始WavLM特征。
原文摘要 · Abstract (English)
Backchannels (e.g., `yeah', `mhm', and `right') are short, non-interruptive feedback signals whose lexical form and prosody jointly convey pragmatic meaning. While prior computational research has largely focused on predicting backchannel timing, the relationship between lexico-prosodic form and meaning remains underexplored. We propose a two-stage framework: first, fine-tuning large language models on dialogue transcripts to derive rich contextual representations; and second, learning a joint embedding space for dialogue contexts and backchannel realizations. We evaluate alignment with human perception via triadic similarity judgments (prosodic and cross-lexical) and a context-backchannel suitability task. Our results demonstrate that the learned projections substantially improve context-backchannel retrieval compared to previous methods. In addition, they reveal that backchannel form is highly sensitive to extended conversational context and that the learned embeddings align more closely with human judgments than raw WavLM features.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。