AI回应用户时需匹配其专业水平，否则体验变差。
Speaking the Right Language: The Impact of Expertise Alignment in User-AI Interactions
- AI在77%对话中以专家级水平回应
- 用户专业水平与AI回应不匹配时体验下降，复杂任务更明显
- 匹配用户水平的回应能增加互动字数，提升参与感
基于25,000条Bing Copilot对话样本，我们研究了AI代理在不同领域知识水平用户下的响应表现及其对用户体验的影响。结果显示，在多个主题领域中，代理主要以熟练或专家级别回应（77%的对话），且这一特征与积极用户体验正相关，无论用户自身专业水平如何。当代理回应水平低于用户时，即出现专业度错配，整体用户体验显著下降，尤其在复杂任务中影响更严重。此外，当代理回应水平与用户匹配时，用户会投入更多语言交互（表现为对话字数增加）。研究强调，在设计以人为本的AI系统时，需重视用户与AI之间的专业能力对齐，以保障高效、满意的交互体验。
原文摘要 · Abstract (English)
Using a sample of 25,000 Bing Copilot conversations, we study how the agent responds to users of varying levels of domain expertise and the resulting impact on user experience along multiple dimensions. Our findings show that across a variety of topical domains, the agent largely responds at proficient or expert levels of expertise (77% of conversations) which correlates with positive user experience regardless of the user's level of expertise. Misalignment, such that the agent responds at a level of expertise below that of the user, has a negative impact on overall user experience, with the impact more profound for more complex tasks. We also show that users engage more, as measured by the number of words in the conversation, when the agent responds at a level of expertise commensurate with that of the user. Our findings underscore the importance of alignment between user and AI when designing human-centered AI systems, to ensure satisfactory and productive interactions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。