解决大模型对话中过度使用人设的问题，提升对话自然度
Controlling and Assessing Appropriate Persona Use in LLM-based Dialogue Generation
- 通过干预提示编码阶段的内部表示抑制人设滥用
- 提出新评估指标，同时惩罚过度和不足使用人设
- 适合关注对话自然性和可控性的研究人员
在基于人设的对话生成中，大模型常不顾上下文地过度使用人设属性，导致回应不自然。尽管这一问题具有实际意义，但其根本原因尚未明确，且缺乏有效缓解方法和评估指标。我们对基于大模型的人设对话生成进行了全面分析，发现大模型存在系统性倾向，会无差别地使用所有给定的人设属性，且现有指标无法捕捉人设使用的上下文合理性。基于此，我们提出自对比人设滥用抑制方法（SCONPOS），通过在提示编码阶段直接干预大模型内部表示，无需生成回复即可抑制过用。同时提出人设恰当性评分（PAS），该指标对过度和不足使用均施加惩罚。实验表明，SCONPOS可系统性减少人设过用，PAS能有效衡量人设使用的上下文恰当性。
原文摘要 · Abstract (English)
In persona-based dialogue generation (PDG), LLMs often overuse persona attributes by incorporating them regardless of dialogue context, resulting in unnatural responses. Despite its practical significance, the underlying causes remain unexplored, with no method to mitigate this problem or metric to assess the appropriateness of persona use. To address these issues, we first conduct a comprehensive analysis of LLM-based PDG, revealing that LLMs exhibit a systematic bias to incorporate all given persona attributes, and that existing metrics fail to capture contextual appropriateness. Building on these findings, we propose Self-CONtrastive Persona Overuse Suppression (SCONPOS) to mitigate overuse by directly intervening in LLMs' internal representations at the prompt encoding stage, without requiring any response generation. We further propose the Persona Appropriateness Score (PAS), a novel metric that penalizes both overuse and underuse. Experimental results demonstrate that SCONPOS systematically reduces overuse, and PAS captures the contextual appropriateness of persona use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。