用响应性量化对话质量,识别建设性交流的关键特征
Computational Analysis of Conversation Dynamics through Participant Responsivity
- 通过语义相似度和大模型分析对话轮次间的响应关系
- 人工标注数据验证下,大模型方法表现更优
- 可区分不同对话的互动结构,适合研究社交对话质量
现有研究多关注话语中的毒性与极化,较少关注对话为何具有建设性。本文提出以“响应性”为核心——即当前发言是否回应前一发言。通过语义相似性及先进大语言模型(LLM)识别发言间关系,构建响应性量化方法,并在人工标注的对话数据集上进行评估。进一步,基于表现更优的LLM方法,分析回应是否具有实质性。将响应性链接视为对话本质特征,发现对话间响应结构差异显著。据此开发对话层面衍生指标,用于刻画多样对话,结果表明这些指标能有效区分不同对话的互动模式。
原文摘要 · Abstract (English)
Growing literature explores toxicity and polarization in discourse, with comparatively less work on characterizing what makes dialogue prosocial and constructive. We explore conversational discourse and investigate a method for characterizing its quality built upon the notion of ``responsivity'' -- whether one person's conversational turn is responding to a preceding turn. We develop and evaluate methods for quantifying responsivity -- first through semantic similarity of speaker turns, and second by leveraging state-of-the-art large language models (LLMs) to identify the relation between two speaker turns. We evaluate both methods against a ground truth set of human-annotated conversations. Furthermore, selecting the better performing LLM-based approach, we characterize the nature of the response -- whether it responded to that preceding turn in a substantive way or not. We view these responsivity links as a fundamental aspect of dialogue but note that conversations can exhibit significantly different responsivity structures. Accordingly, we then develop conversation-level derived metrics to address various aspects of conversational discourse. We use these derived metrics to explore other conversations and show that they support meaningful characterizations and differentiations across a diverse collection of conversations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。