让对话模型同时理解自己和对方的人设,生成更自然的互动。
When Harry Meets Superman: The Role of The Interlocutor in Persona-Based Dialogue Generation
- 同时感知自身与对话人设,提升回应一致性
- 熟悉对方时表现更好,陌生对方则效果下降
- 适合研究人物角色对话、社交智能系统的开发者
赋予对话代理人格信息可显著提升其生成内容的一致性与多样性。尽管已有大量研究聚焦于使对话与给定人格一致,但对对话对象个人特征的适应仍鲜有探索。本文研究三个关键问题:(1) 模型同时对齐自身人格与对话对象的能力;(2) 在熟悉或不熟悉对话者与话题下的鲁棒性;(3) 针对特定人格对话进行额外微调的影响。通过多种说话人组合与话题评估,将评价任务设定为作者身份识别,并结合大模型评判与人工评估。系统性地屏蔽或披露对话对象信息,以评估其对生成结果的影响。结果显示,知晓对方人格可提高目标说话人识别率,屏蔽则相反;模型在不同话题间泛化良好,但在面对陌生对话者时表现不佳;零样本设置下,大模型常复制传记细节,虽利于识别但使任务变得简单。
原文摘要 · Abstract (English)
Endowing dialogue agents with persona information has proven to significantly improve the consistency and diversity of their generations. While much focus has been placed on aligning dialogues with provided personas, the adaptation to the interlocutor's profile remains largely underexplored. In this work, we investigate three key aspects: (1) a model's ability to align responses with both the provided persona and the interlocutor's; (2) its robustness when dealing with familiar versus unfamiliar interlocutors and topics, and (3) the impact of additional fine-tuning on specific persona-based dialogues. We evaluate dialogues generated with diverse speaker pairings and topics, framing the evaluation as an author identification task and employing both LLM-as-a-judge and human evaluations. By systematically masking or disclosing information about the interlocutor, we assess its impact on dialogue generation. Results show that access to the interlocutor's persona improves the recognition of the target speaker, while masking it does the opposite. Although models generalise well across topics, they struggle with unfamiliar interlocutors. Finally, we found that in zero-shot settings, LLMs often copy biographical details, facilitating identification but trivialising the task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。