用九种心理语言特征构建可控情感求助者模拟器，提升评估真实性。
Stress-Testing Emotional Support Models: Moving from Homogeneous to Diverse Help Seekers
- 基于九类心理语言特征，用MoE架构训练可调控的求助者模拟器。
- 相比现有方法，行为多样性与角色贴合度显著提升。
- 适合评估情感聊天机器人在真实复杂求助场景下的表现。
随着情感支持聊天机器人在研究和产业中日益流行，一种常见评估策略是使用模拟求助者与支持型聊天机器人互动。然而，当前模拟器存在两大缺陷：(1) 无法捕捉真实求助者的多样化行为，常将其表现为过度配合；(2) 缺乏对特定求助者画像的可控性。为此，我们提出一个基于九种心理与语言特征的可控求助者模拟器。利用真实的Reddit对话数据，通过混合专家（MoE）架构进行训练，有效将不同求助行为划分为专用参数子空间，从而实现细粒度可控性。该模拟器在角色贴合度与行为多样性方面均优于现有方法。使用该系统评估7个主流支持模型时，揭示了此前未被发现的性能下降现象。这些结果凸显了本框架在提供更真实、更具压力测试性的评估方面的价值。
原文摘要 · Abstract (English)
As emotional support chatbots have recently gained significant traction across both research and industry, a common evaluation strategy has emerged: use help-seeker simulators to interact with supporter chatbots. However, current simulators suffer from two critical limitations: (1) they fail to capture the behavioral diversity of real-world seekers, often portraying them as overly cooperative, and (2) they lack the controllability required to simulate specific seeker profiles. To address these challenges, we present a controllable seeker simulator driven by nine psychological and linguistic features that underpin seeker behavior. Using authentic Reddit conversations, we train our model via a Mixture-of-Experts (MoE) architecture, which effectively differentiates diverse seeker behaviors into specialized parameter subspaces, thereby enhancing fine-grained controllability. Our simulator achieves superior profile adherence and behavioral diversity compared to existing approaches. Furthermore, evaluating 7 prominent supporter models with our system uncovers previously obscured performance degradations. These findings underscore the utility of our framework in providing a more faithful and stress-tested evaluation for emotional support chatbots.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。